-- Logs begin at Tue 2022-07-05 15:10:44 UTC, end at Tue 2022-07-05 16:32:16 UTC. -- Jul 05 15:49:25.239677 dv-cinder-848759-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 05 15:49:25.248711 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 05 15:49:25.250687 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 5 15:49:25 2022] *** Jul 05 15:49:25.250687 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 05 15:49:25.250687 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 05 15:49:25.250687 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: nodename: dv-cinder-848759-1-iscsi Jul 05 15:49:25.250687 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: machine: x86_64 Jul 05 15:49:25.250687 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: clock source: unix Jul 05 15:49:25.250687 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: pcre jit disabled Jul 05 15:49:25.250687 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: detected number of CPU cores: 4 Jul 05 15:49:25.250687 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: current working directory: / Jul 05 15:49:25.250687 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: detected binary path: /usr/bin/uwsgi-core Jul 05 15:49:25.251244 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: your processes number limit is 47680 Jul 05 15:49:25.251244 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: your memory page size is 4096 bytes Jul 05 15:49:25.251244 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: detected max file descriptor number: 2048 Jul 05 15:49:25.251244 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: lock engine: pthread robust mutexes Jul 05 15:49:25.251244 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: thunder lock: enabled Jul 05 15:49:25.251244 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 05 15:49:25.251244 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 05 15:49:25.294411 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: Python main interpreter initialized at 0x5633a915f9c0 Jul 05 15:49:25.294411 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: python threads support enabled Jul 05 15:49:25.294411 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: your server socket listen backlog is limited to 100 connections Jul 05 15:49:25.294411 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: your mercy for graceful operations on workers is 90 seconds Jul 05 15:49:25.294751 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: mapped 403077 bytes (393 KB) for 2 cores Jul 05 15:49:25.294853 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: *** Operational MODE: preforking *** Jul 05 15:49:25.295130 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: *** uWSGI is running in multiple interpreter mode *** Jul 05 15:49:25.295130 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: spawned uWSGI master process (pid: 80385) Jul 05 15:49:25.295330 dv-cinder-848759-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 05 15:49:25.296413 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: spawned uWSGI worker 1 (pid: 80397, cores: 1) Jul 05 15:49:25.296413 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: spawned uWSGI worker 2 (pid: 80398, cores: 1) Jul 05 15:49:25.296413 dv-cinder-848759-1-iscsi devstack@keystone.service[80385]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 05 15:49:30.754944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 15:49:30.754944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: from cryptography.utils import int_from_bytes Jul 05 15:49:30.755695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 15:49:30.755695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: from cryptography.utils import int_from_bytes Jul 05 15:49:30.755827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 15:49:30.755827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: from cryptography.utils import int_from_bytes Jul 05 15:49:30.756250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 15:49:30.756250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: from cryptography.utils import int_from_bytes Jul 05 15:49:30.803788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 05 15:49:30.804531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 05 15:49:30.804745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] command line args: [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 05 15:49:30.804879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 05 15:49:30.805040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 05 15:49:30.805117 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 05 15:49:30.805241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] command line args: [] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 05 15:49:30.805372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ================================================================================ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 05 15:49:30.805507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 05 15:49:30.805596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] admin_token = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.805676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ================================================================================ {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 05 15:49:30.805781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config_dir = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.805921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] admin_token = **** {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.806005 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.806163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] config_dir = [] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.806248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] config_source = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.806347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.806442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] debug = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.806522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] config_source = [] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.806598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.806841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] debug = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.806910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.806997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.807125 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.807208 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.807272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.807447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.807540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.807734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.807816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.807963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] list_limit = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.808048 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.808150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] log_config_append = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.808235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_config_append = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.808336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.808409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.808506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] log_dir = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.808584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_dir = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.808683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] log_file = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.808788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.808924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.809017 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.809158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.809240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.809403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.809491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.809649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.809731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.809821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.809947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.810036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.810112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.810224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.810321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.810475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.810560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.810692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.810804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.810933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.811018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.811145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.811336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.811449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.811697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.811802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.812045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.812045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.812374 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.812553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.812697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.812697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.813069 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.813308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] publish_errors = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.813414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] publish_errors = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.813650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.813819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.813819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.814097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.814240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.814361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.814498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.814630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.814755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.814898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.815020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] standard_threads = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.815132 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] standard_threads = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.815267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.815386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.815498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.815638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.815758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.815876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.815998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_journal = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.816111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] use_journal = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.816221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_json = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.816338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] use_json = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.816449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_stderr = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.816559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] use_stderr = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.816745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] use_syslog = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.816887 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] use_syslog = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.817028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.817140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 15:49:30.817250 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.817360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.817484 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.817603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.817715 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.817826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.817936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.818048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.818167 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.818289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.818407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.818519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.818636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.818747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.818866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.818977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.819089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.819205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.819318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.819438 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.820213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.820333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.820445 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.820555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.820672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.820837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.820965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.821078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.821196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.821308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.821425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.821537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.821657 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.821769 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.821887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.822002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.822120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.822246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.822365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.822477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.822588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.822698 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.822809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.822996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.822996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.823208 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.823326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.823438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.823549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.823668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.823782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.823898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.824010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.824120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.824237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.824357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.824470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.824581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.824697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.824840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.824954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.825077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.825196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.825308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.825419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.825529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.825646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.825766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.825766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.825975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.826086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.826203 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.826320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.826429 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.826525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.826618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.826618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.826824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.826929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.827038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.827142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.827237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.827330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.827424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.827551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.827669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.827669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.827880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.827982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.828092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.828209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.828321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.828440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.828565 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.828679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.828832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.828941 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.829052 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.829173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.829277 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.829387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.829499 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.829620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.829740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.829861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.829975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.830087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.830198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.830310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.830422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.830542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.830542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.830754 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.830866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.830974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.831085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.831205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.831308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.831425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.831538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.831640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.831757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.831878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.831995 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.832109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.832229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.832350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.832472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.832594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.832709 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.832852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.832966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.833078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.833189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.833313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.833428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.833540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.833652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.833765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.833877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.834064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.834180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.834293 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.834405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.834517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.834629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.834748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.834863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.834975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.835109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.835226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.835348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.835471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.835593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.835707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.835825 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.835939 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.836051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.836169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.836290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.836404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.836522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.836641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.836755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.836912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.837040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.837161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.837274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.837387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.837499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.837611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.837731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.837845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.837958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.838070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.838188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.838301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.838429 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.838543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.838656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.838768 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.838880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.838993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.839114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.839227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.839339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.839451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.839564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.839676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.839796 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.839910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.840022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.840135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.840255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.840369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.840489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.840603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.840716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.840852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.840965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.841077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.841206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.841321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.841432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.841550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.841668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.841781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.841901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.external = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.842018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.842132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.842244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] auth.external = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.842356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.842475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.842619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.842619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.password = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.842851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.842966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] auth.token = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.843085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.843199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.843312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] auth.password = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.843425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.843546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] auth.token = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.843660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.843781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.843895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.844008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.844126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.844249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.844364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.844482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.844595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.844707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.844869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.844995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.845111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.845224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.845337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.845451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] credential.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.845564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.845685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.845800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.845919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.846033 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.846145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.846258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.846380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.846495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.846608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.846721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.846842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.846957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.847078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.847204 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.847319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.847433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.847546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.847659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.847850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.847966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.848079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.848199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.848313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.848433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.848556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.848670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.848801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.848917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.849030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.849142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.849267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] federation.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.849382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.849496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.849609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.849722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.849842 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.849957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.850070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.850183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.850295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.850409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.850531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.850646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.851272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.851637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.851759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.851874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.851999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.852113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.852227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.852340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.852464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.852584 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.852707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.852870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.852985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.853098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.853210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.853322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.853447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.853562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.853677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.853789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.853901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.854013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.854137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.854137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.854351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.854469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.854583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.854695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.854817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.854931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.855044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.855155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.855267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.855380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.855501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.855615 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.855727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.855841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.855954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.856067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.856188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.856302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.856414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.856526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.856641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.856753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.856899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.857015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.857127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.857239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.857352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.857464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.857589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.857705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.857818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.857932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.858045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.858158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.858279 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.858393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.858505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.858617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.858729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.858850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.858965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.859078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.859191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.859309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.859423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.859544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.859658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.859772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.859884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.859998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.860110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.860231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.860345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.860458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.860570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.860694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.860851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.860978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.861095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.861208 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.861321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.861433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.861544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.861665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.861780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.861893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.862005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.862120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.862233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.862354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.862468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.862586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.862698 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.862811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.862924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.863048 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.863164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.863277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.863390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.863503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.863616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.863737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.863851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.863964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.864077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.864191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.864303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.864424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.864539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.864652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.864778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.864904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.865019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.865139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.865255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.865368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.865481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.865594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.865707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.865827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.865942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.866055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.866055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.866268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.866428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.867051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.867169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.867283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.867430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.867544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.867544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.867766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.867881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.867994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.868107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.868219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.868332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.868454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.868573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.868687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.868687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.868943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.869057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.869183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.869299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.869413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.869526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.869639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.869752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.869873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.869989 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.870101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.870214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.870326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.870447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.870569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.870684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.870798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.870910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.871023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.871136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.871319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.871436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.871551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.871663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.871776 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.871888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.872010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.872124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.872238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.872350 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.872464 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.872581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.872703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.872841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.872957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.873070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.873183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.873295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.873416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.873531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.873645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.873758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.873871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.873984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.874104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] resource.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.874219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.874333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.874445 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.874558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.874680 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.874796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.874909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.875022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.875134 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.875253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.875376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.875497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.875612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.875732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.875847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.875959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.876081 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.876196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.876309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.876423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] role.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.876535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.876647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] role.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.876815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.876933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.877046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.877160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.877273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.877385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.877510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.877626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.877739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.877853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.877966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.878078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.878199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.878313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.878426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.878539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.878651 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.878764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.878886 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.879000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.879000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.879211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.879211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.879422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.879543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.879658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.879772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.879885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.879998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.880111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.880232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.880347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.880460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.880572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.880685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.880829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.880948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.881062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.881175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.881287 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.881400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.881521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.881636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.881750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.881863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.881977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.882090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.882211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.882325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.882325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.882541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.882655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.882768 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.882889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.883004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.883118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.883231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.883344 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.883457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.883578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.883693 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.883805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.883918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.884031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.884145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.884266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.884381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.884493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.884607 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] token.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.884719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.884857 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.884987 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.885103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.885218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.885331 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.885443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.885556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.885677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.885791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.885904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.886018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.886131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.886245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.886366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.886489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.886603 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.886716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.886829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.886942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.887063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.887178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.887293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.887406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.887518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.887640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.887763 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.887889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.888004 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.888118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.888231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.888345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.888458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.888579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.888695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.888846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.connection = **** {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.888974 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.889098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.889212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.889325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.889438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.889625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.889741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.889856 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.889968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.890081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.890194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.890315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.890431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.890545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.890660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.890773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.890886 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.891007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.891122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.891236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.891349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.891464 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.891577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.891698 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.891813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.891813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 05 15:49:30.892035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.892035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.892035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.892035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.892035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.892035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 15:49:30.892035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 05 15:49:30.919526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.919759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.919880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.919980 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.920067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.920220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.928354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.928596 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.968938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.969164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.969749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:30.969980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.113133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.114450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.162440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.162612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.162690 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.162847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.228524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.228748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.228927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.229014 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.229159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.229244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.229381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.229463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.229587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.229678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.229802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.230015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.253332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.253557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.253641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.253778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.253862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.253997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.254079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.254212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.254305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.254424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.254508 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.254635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.254719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.254848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.265717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.266031 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.278964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.279225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.279411 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.279495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.279628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.279712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.279841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.279924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.280050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.280135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.280258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.280341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.280499 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.280713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 15:49:31.298129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.298337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 15:49:31.298668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.298796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 15:49:31.299065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 15:49:31.299190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.299316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.299439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.299570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 15:49:31.299700 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.299840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.299969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.300051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.300124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.300354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.310661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.310941 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.311195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.311276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.311490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.311576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.311741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.312019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.319716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.320337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.323001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.323277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.323553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.323634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.323829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.323922 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.324104 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.324185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.324357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.324437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.324618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.324700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.324907 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.324990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.325171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.325262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.325434 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.325514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.325692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.325771 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.325971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.326234 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.353294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.354216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.354892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.355228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.356020 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.356301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.360503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.360782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.361184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.361602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.361878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.362155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.368499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.368735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.369636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.369913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.373199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.373480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.373755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.374027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.374291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.374409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.374555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.374691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.374816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.374970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.375074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.375248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.375519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.375786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.376049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.376309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.393638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.393907 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.395393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.395664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.398942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.400825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.402747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.403038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.404481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.404786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.408582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.409982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.412194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.414237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.416455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.416743 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.417050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.417333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.417604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.417874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.417967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.418196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.418488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.418778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.419059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.419333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.436859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.437138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.437405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.437662 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.438406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.438686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.438948 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.439204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.445562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.447159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.447268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.447544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.448896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.449170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.450736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.451009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.451283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.451560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.451847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.452336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.452616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.452925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.453204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.453479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.465793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.466068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.466344 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.466617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.466889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.467289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.467568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.467842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.468115 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.468390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.468551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.468677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.468860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.468976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.487446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.489207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.490727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.492502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.494057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.495781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.495875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.497427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.497513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.497701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.497982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.499216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.499492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.499778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.505257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.506841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.507221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.508846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.510311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.510594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.510978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.511253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.512076 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.512359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.512635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.512930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.520349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.520629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.520922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.522153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.522437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.523238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.529292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.529567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.529838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.530103 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.530363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.530741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.531012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.531151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 15:49:31.531283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.531428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.531556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.531704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.531829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.531976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.532099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.532242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.532353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.532514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.532597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.532801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.532887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.533081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.533454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.533732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.534008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.534270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.534532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.534793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 15:49:31.570893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5633a915f9c0 pid: 80398 (default app) Jul 05 15:49:31.573089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5633a915f9c0 pid: 80397 (default app) Jul 05 15:49:31.574209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a553c87-9c80-446b-b89a-ab2f64a23310 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:31.574499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a553c87-9c80-446b-b89a-ab2f64a23310 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:31.574774 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a553c87-9c80-446b-b89a-ab2f64a23310 None None] PATH_INFO: `/v3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:31.578916 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Tue Jul 5 15:49:31 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:40.042186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62039c57-197d-4e51-abc3-55872b2081cd None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:40.042748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62039c57-197d-4e51-abc3-55872b2081cd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:40.042748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62039c57-197d-4e51-abc3-55872b2081cd None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:40.047000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1/2] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:40.052489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:40.052902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:40.053090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:40.130360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: INFO dbcounter [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] Registered counter for database keystone Jul 05 15:49:40.155865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG oslo_db.sqlalchemy.engines [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80398) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 15:49:40.162133 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writer thread running {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 15:49:40.346294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG passlib.handlers.bcrypt [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80398) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 05 15:49:40.346622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG passlib.handlers.bcrypt [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80398) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 05 15:49:40.402594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:40.404283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:40.530354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 05 15:49:40.730831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62039c57-197d-4e51-abc3-55872b2081cd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:40.808783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: INFO dbcounter [None req-62039c57-197d-4e51-abc3-55872b2081cd None None] Registered counter for database keystone Jul 05 15:49:40.828334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG oslo_db.sqlalchemy.engines [None req-62039c57-197d-4e51-abc3-55872b2081cd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80397) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 15:49:40.834545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writer thread running {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 15:49:40.953583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62039c57-197d-4e51-abc3-55872b2081cd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:40.957783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7a2a1-804f-4ff2-a06f-3a5901cabb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:40.958485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a2a1-804f-4ff2-a06f-3a5901cabb71 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:40.958773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a2a1-804f-4ff2-a06f-3a5901cabb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:40.959045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a2a1-804f-4ff2-a06f-3a5901cabb71 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:40.967616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7a2a1-804f-4ff2-a06f-3a5901cabb71 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:41.019724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9aa7a2a1-804f-4ff2-a06f-3a5901cabb71 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 05 15:49:41.020046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7a2a1-804f-4ff2-a06f-3a5901cabb71 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:41.030750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1324 bytes} [Tue Jul 5 15:49:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:49:41.036682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:41.051695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aaa99-aff2-4f3d-942e-45c23aca3f0c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:41.056132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e6f4e-6169-4947-babf-c799f57fa098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:41.057275 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6f4e-6169-4947-babf-c799f57fa098 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:41.057568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6f4e-6169-4947-babf-c799f57fa098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:41.057846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6f4e-6169-4947-babf-c799f57fa098 admin admin] PATH_INFO: `/v3/projects/35018c3fbb4f440c9fd930631a71e6c6` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:41.059337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e6f4e-6169-4947-babf-c799f57fa098 admin admin] RBAC: Authorizing `identity:get_project(project_id=35018c3fbb4f440c9fd930631a71e6c6)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:41.114560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d90e6f4e-6169-4947-babf-c799f57fa098 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 05 15:49:41.114904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e6f4e-6169-4947-babf-c799f57fa098 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:41.119588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 3/5] 10.222.0.45 () {56 vars in 1207 bytes} [Tue Jul 5 15:49:41 2022] GET /identity/v3/projects/35018c3fbb4f440c9fd930631a71e6c6 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:43.099986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924c7fe-ad43-4a41-9a4e-fd1ff8e55cb5 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:43.100524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924c7fe-ad43-4a41-9a4e-fd1ff8e55cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:43.100617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924c7fe-ad43-4a41-9a4e-fd1ff8e55cb5 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:43.104948 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 3/6] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:49:43.111608 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655ff58-570d-40a4-ab30-268096f7787c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:43.112056 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655ff58-570d-40a4-ab30-268096f7787c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:43.112181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655ff58-570d-40a4-ab30-268096f7787c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:43.141368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6655ff58-570d-40a4-ab30-268096f7787c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:43.142885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6655ff58-570d-40a4-ab30-268096f7787c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:43.151357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 4/7] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 05 15:49:43.173311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924c7fe-ad43-4a41-9a4e-fd1ff8e55cb5 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:43.185270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924c7fe-ad43-4a41-9a4e-fd1ff8e55cb5 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:43.189386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba80180-1971-441c-bcfc-054826fe60e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:43.190017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba80180-1971-441c-bcfc-054826fe60e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:43.190303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba80180-1971-441c-bcfc-054826fe60e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:43.190581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba80180-1971-441c-bcfc-054826fe60e8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:43.198712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba80180-1971-441c-bcfc-054826fe60e8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:43.200490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba80180-1971-441c-bcfc-054826fe60e8 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:43.209250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 4/8] 10.222.0.45 () {58 vars in 1324 bytes} [Tue Jul 5 15:49:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:49:43.215085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655ff58-570d-40a4-ab30-268096f7787c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:43.224085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655ff58-570d-40a4-ab30-268096f7787c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:43.228711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0123c-33a6-449a-8d6a-7c940652c323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:43.229597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0123c-33a6-449a-8d6a-7c940652c323 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:43.229890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0123c-33a6-449a-8d6a-7c940652c323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:43.230170 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0123c-33a6-449a-8d6a-7c940652c323 admin admin] PATH_INFO: `/v3/users/45ceaef67f814339b45f2d297054bedf` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:43.231568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0123c-33a6-449a-8d6a-7c940652c323 admin admin] RBAC: Authorizing `identity:get_user(user_id=45ceaef67f814339b45f2d297054bedf)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:43.233443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0123c-33a6-449a-8d6a-7c940652c323 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:43.235018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 5/9] 10.222.0.45 () {56 vars in 1198 bytes} [Tue Jul 5 15:49:43 2022] GET /identity/v3/users/45ceaef67f814339b45f2d297054bedf => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:45.331903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c57b29-b890-4dc4-9e30-f4f2e1ffb70a None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:45.332388 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c57b29-b890-4dc4-9e30-f4f2e1ffb70a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:45.332484 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c57b29-b890-4dc4-9e30-f4f2e1ffb70a None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:45.333832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 5/10] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:49:45.336117 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfcfc5-4c33-4cc6-8ea7-74a5976fbe47 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:45.336488 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfcfc5-4c33-4cc6-8ea7-74a5976fbe47 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:45.336677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfcfc5-4c33-4cc6-8ea7-74a5976fbe47 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:45.338125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 6/11] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:45.338621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeeff19-158d-495e-8c1a-d8e2af880030 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:45.338921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeeff19-158d-495e-8c1a-d8e2af880030 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:45.339210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeeff19-158d-495e-8c1a-d8e2af880030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:45.343065 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccefe7b-582b-4d4b-b7d3-5d0469493465 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:45.343351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccefe7b-582b-4d4b-b7d3-5d0469493465 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:45.343635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccefe7b-582b-4d4b-b7d3-5d0469493465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:45.360540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5ccefe7b-582b-4d4b-b7d3-5d0469493465 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:45.361752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5ccefe7b-582b-4d4b-b7d3-5d0469493465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:45.379966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG passlib.handlers.bcrypt [None req-fbeeff19-158d-495e-8c1a-d8e2af880030 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80397) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 05 15:49:45.380389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG passlib.handlers.bcrypt [None req-fbeeff19-158d-495e-8c1a-d8e2af880030 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80397) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 05 15:49:45.426066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-fbeeff19-158d-495e-8c1a-d8e2af880030 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:45.427624 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fbeeff19-158d-495e-8c1a-d8e2af880030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:45.442242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 7/12] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:49:45.465788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccefe7b-582b-4d4b-b7d3-5d0469493465 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:45.489938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:45.527836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 6/13] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:49:45.550348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeeff19-158d-495e-8c1a-d8e2af880030 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:45.570142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccefe7b-582b-4d4b-b7d3-5d0469493465 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:45.604279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a1f67-7fbb-498a-8c12-9b234fadd77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:45.605100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a1f67-7fbb-498a-8c12-9b234fadd77b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:45.605391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a1f67-7fbb-498a-8c12-9b234fadd77b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:45.605662 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a1f67-7fbb-498a-8c12-9b234fadd77b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:45.616814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a1f67-7fbb-498a-8c12-9b234fadd77b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e74a1f67-7fbb-498a-8c12-9b234fadd77b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:49:45.627929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:49:45.628854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:49:45.629933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 8/14] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:49:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:49:45.634902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a1f67-7fbb-498a-8c12-9b234fadd77b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:45.650108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeeff19-158d-495e-8c1a-d8e2af880030 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:45.652373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:45.687236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b6299-0411-45cd-9edb-11c8d1d7d93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:45.688141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b6299-0411-45cd-9edb-11c8d1d7d93a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:45.688620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b6299-0411-45cd-9edb-11c8d1d7d93a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:45.688983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b6299-0411-45cd-9edb-11c8d1d7d93a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:45.716264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:45.734845 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a1f67-7fbb-498a-8c12-9b234fadd77b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:45.757006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b6299-0411-45cd-9edb-11c8d1d7d93a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:45.758559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b6299-0411-45cd-9edb-11c8d1d7d93a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:45.768010 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b181a6-5e52-46d5-ae13-d47e533c26ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:45.768665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b181a6-5e52-46d5-ae13-d47e533c26ad None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:45.768976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b181a6-5e52-46d5-ae13-d47e533c26ad None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:45.769251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b181a6-5e52-46d5-ae13-d47e533c26ad None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:45.769993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b181a6-5e52-46d5-ae13-d47e533c26ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:49:45.770296 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b181a6-5e52-46d5-ae13-d47e533c26ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:45.771670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b181a6-5e52-46d5-ae13-d47e533c26ad None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:45.783792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 9/15] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:49:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:49:45.790171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b181a6-5e52-46d5-ae13-d47e533c26ad None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:45.856724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 7/16] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:49:45.862135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b6299-0411-45cd-9edb-11c8d1d7d93a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:45.864787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:45.889958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b181a6-5e52-46d5-ae13-d47e533c26ad None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:45.923763 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c2ff9-8c8d-4732-baa1-fd4dffdccd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:45.924634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c2ff9-8c8d-4732-baa1-fd4dffdccd1a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:45.924956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c2ff9-8c8d-4732-baa1-fd4dffdccd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:45.925242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c2ff9-8c8d-4732-baa1-fd4dffdccd1a None admin] PATH_INFO: `/v3/users/45ceaef67f814339b45f2d297054bedf` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:45.926466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c2ff9-8c8d-4732-baa1-fd4dffdccd1a None admin] RBAC: Authorizing `identity:get_user(user_id=45ceaef67f814339b45f2d297054bedf)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:45.927951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c2ff9-8c8d-4732-baa1-fd4dffdccd1a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:45.929625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 10/17] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:49:45 2022] GET /identity/v3/users/45ceaef67f814339b45f2d297054bedf => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:49:45.935049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c2ff9-8c8d-4732-baa1-fd4dffdccd1a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:45.950141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:45.960672 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b6299-0411-45cd-9edb-11c8d1d7d93a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:45.994105 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c69bec-51a0-4081-8311-7ce6319ef476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:45.994896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c69bec-51a0-4081-8311-7ce6319ef476 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:45.995181 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c69bec-51a0-4081-8311-7ce6319ef476 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:45.995460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c69bec-51a0-4081-8311-7ce6319ef476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:46.014086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c69bec-51a0-4081-8311-7ce6319ef476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:46.015571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c69bec-51a0-4081-8311-7ce6319ef476 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:46.033577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c2ff9-8c8d-4732-baa1-fd4dffdccd1a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-b9c69bec-51a0-4081-8311-7ce6319ef476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:49:46.036632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:49:46.037439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:49:46.038380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:49:46.038380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:49:46.038380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:49:46.038380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:49:46.038380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 8/18] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:49:46.043545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c69bec-51a0-4081-8311-7ce6319ef476 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:46.067047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a678b-0dfe-4f41-ac98-b42fe05db99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:46.067840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a678b-0dfe-4f41-ac98-b42fe05db99b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:46.068134 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a678b-0dfe-4f41-ac98-b42fe05db99b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:46.068413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a678b-0dfe-4f41-ac98-b42fe05db99b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:46.070148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a678b-0dfe-4f41-ac98-b42fe05db99b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:46.071614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a678b-0dfe-4f41-ac98-b42fe05db99b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:46.073258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 11/19] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:49:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:49:46.078783 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a678b-0dfe-4f41-ac98-b42fe05db99b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:46.099952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:46.143556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c69bec-51a0-4081-8311-7ce6319ef476 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:46.159455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:46.176122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b04bfc-a6e0-40cd-bb19-befdf7dca2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:46.176789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b04bfc-a6e0-40cd-bb19-befdf7dca2dc None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:46.177081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b04bfc-a6e0-40cd-bb19-befdf7dca2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:46.177354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b04bfc-a6e0-40cd-bb19-befdf7dca2dc None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:46.177678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a678b-0dfe-4f41-ac98-b42fe05db99b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:46.178069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b04bfc-a6e0-40cd-bb19-befdf7dca2dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:49:46.178373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b04bfc-a6e0-40cd-bb19-befdf7dca2dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:46.179789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b04bfc-a6e0-40cd-bb19-befdf7dca2dc None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:46.197255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 9/20] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:49:46.212670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca438e-a79c-4c87-af8d-1adea66101d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:46.213402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca438e-a79c-4c87-af8d-1adea66101d2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:46.213691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca438e-a79c-4c87-af8d-1adea66101d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:46.213971 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca438e-a79c-4c87-af8d-1adea66101d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:46.214838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca438e-a79c-4c87-af8d-1adea66101d2 None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.domain.id=default, user.id=45ceaef67f814339b45f2d297054bedf) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:49:46.215158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca438e-a79c-4c87-af8d-1adea66101d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:46.216663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca438e-a79c-4c87-af8d-1adea66101d2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:46.224473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 12/21] 10.222.0.45 () {56 vars in 1322 bytes} [Tue Jul 5 15:49:46 2022] GET /identity/v3/role_assignments?user.id=45ceaef67f814339b45f2d297054bedf&scope.domain.id=default&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:49:48.385933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df158b-5049-4cd1-8ad6-44f05db67cde None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:48.386896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df158b-5049-4cd1-8ad6-44f05db67cde None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:48.387393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df158b-5049-4cd1-8ad6-44f05db67cde None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:48.389005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 10/22] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:48.394215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9caab-081d-411f-b454-1b3b86c2b1c0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:48.395037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9caab-081d-411f-b454-1b3b86c2b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:48.395523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9caab-081d-411f-b454-1b3b86c2b1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:48.418085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-75a9caab-081d-411f-b454-1b3b86c2b1c0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:48.419606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-75a9caab-081d-411f-b454-1b3b86c2b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:48.507257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 13/23] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:49:48.548959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df158b-5049-4cd1-8ad6-44f05db67cde None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:48.686640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df158b-5049-4cd1-8ad6-44f05db67cde None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:48.715313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:48.740005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4f28e-a81d-44a1-8a26-f2712a6334d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:48.740851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4f28e-a81d-44a1-8a26-f2712a6334d6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:48.741137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4f28e-a81d-44a1-8a26-f2712a6334d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:48.741414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4f28e-a81d-44a1-8a26-f2712a6334d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:48.758260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4f28e-a81d-44a1-8a26-f2712a6334d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-1ac4f28e-a81d-44a1-8a26-f2712a6334d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:49:48.770628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:49:48.771439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:49:48.772709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 11/24] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:49:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:49:48.782473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9caab-081d-411f-b454-1b3b86c2b1c0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:48.918917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:48.938402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a673d-edcc-4ab5-9d61-1ad8f1d26433 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:48.938933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a673d-edcc-4ab5-9d61-1ad8f1d26433 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:48.939437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a673d-edcc-4ab5-9d61-1ad8f1d26433 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:48.941024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 12/25] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:48.953409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd381a-4c4a-4ce4-86ba-4754f788bbc1 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:48.953581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9caab-081d-411f-b454-1b3b86c2b1c0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:48.954079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd381a-4c4a-4ce4-86ba-4754f788bbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:48.954550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd381a-4c4a-4ce4-86ba-4754f788bbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:48.973629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-fcfd381a-4c4a-4ce4-86ba-4754f788bbc1 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:48.975279 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fcfd381a-4c4a-4ce4-86ba-4754f788bbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:49.017609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db8449-9872-4b95-bc2c-6507019125c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:49.018297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db8449-9872-4b95-bc2c-6507019125c7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.018586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db8449-9872-4b95-bc2c-6507019125c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.018870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db8449-9872-4b95-bc2c-6507019125c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.019640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db8449-9872-4b95-bc2c-6507019125c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:49:49.019956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db8449-9872-4b95-bc2c-6507019125c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:49.021406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db8449-9872-4b95-bc2c-6507019125c7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:49.040099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 14/26] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:49:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:49.046648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db8449-9872-4b95-bc2c-6507019125c7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:49.096228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 13/27] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:49:49.101919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14725813-898f-45cc-b46e-6594ed295633 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.102450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14725813-898f-45cc-b46e-6594ed295633 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.103112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14725813-898f-45cc-b46e-6594ed295633 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.104633 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 14/28] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:49.109417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688348bf-6961-4015-b0ac-ba4c461b44d8 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.110444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688348bf-6961-4015-b0ac-ba4c461b44d8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.111194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688348bf-6961-4015-b0ac-ba4c461b44d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.142282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-688348bf-6961-4015-b0ac-ba4c461b44d8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:49.143826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-688348bf-6961-4015-b0ac-ba4c461b44d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:49.242226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 15/29] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:49:49.244042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688348bf-6961-4015-b0ac-ba4c461b44d8 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:49.283001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db8449-9872-4b95-bc2c-6507019125c7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:49.305646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:49.370204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688348bf-6961-4015-b0ac-ba4c461b44d8 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:49.389456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7805e6e-776b-438e-bd98-3b6e8594ef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:49.390347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7805e6e-776b-438e-bd98-3b6e8594ef46 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.390640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7805e6e-776b-438e-bd98-3b6e8594ef46 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.390921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7805e6e-776b-438e-bd98-3b6e8594ef46 None admin] PATH_INFO: `/v3/users/45ceaef67f814339b45f2d297054bedf` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.397497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7805e6e-776b-438e-bd98-3b6e8594ef46 None admin] RBAC: Authorizing `identity:get_user(user_id=45ceaef67f814339b45f2d297054bedf)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:49.398984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7805e6e-776b-438e-bd98-3b6e8594ef46 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:49.400630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 15/30] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:49:49 2022] GET /identity/v3/users/45ceaef67f814339b45f2d297054bedf => generated 247 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:49:49.401435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7805e6e-776b-438e-bd98-3b6e8594ef46 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:49.412791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:49.413206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e1177-0c78-4f65-aa6e-e588d404906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:49.413942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e1177-0c78-4f65-aa6e-e588d404906a None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.414282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e1177-0c78-4f65-aa6e-e588d404906a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.414565 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e1177-0c78-4f65-aa6e-e588d404906a None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.415323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e1177-0c78-4f65-aa6e-e588d404906a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:49.422501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e1177-0c78-4f65-aa6e-e588d404906a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:49.527002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7805e6e-776b-438e-bd98-3b6e8594ef46 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:49.568422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679f939-e016-44c8-adee-7862bf266323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:49.570338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679f939-e016-44c8-adee-7862bf266323 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.570816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679f939-e016-44c8-adee-7862bf266323 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.571279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679f939-e016-44c8-adee-7862bf266323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.583418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679f939-e016-44c8-adee-7862bf266323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:49.585307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679f939-e016-44c8-adee-7862bf266323 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2679f939-e016-44c8-adee-7862bf266323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:49:49.605770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:49:49.606616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:49:49.607386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:49:49.607386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:49:49.607386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:49:49.607386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:49:49.609353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 16/31] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:49:49.610401 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679f939-e016-44c8-adee-7862bf266323 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:49.671655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:49.679164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 16/32] 10.222.0.45 () {60 vars in 1131 bytes} [Tue Jul 5 15:49:49 2022] POST /identity/v3/roles => generated 219 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:49:49.679164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125da34-1286-4a50-9320-19d2200f7b69 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.679164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125da34-1286-4a50-9320-19d2200f7b69 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.679164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125da34-1286-4a50-9320-19d2200f7b69 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.679164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 17/33] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:49.679958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125da34-1286-4a50-9320-19d2200f7b69 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:49.783512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679f939-e016-44c8-adee-7862bf266323 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:49.870002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72a728-afd8-42e1-a67d-d36aae6143ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:49.870218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72a728-afd8-42e1-a67d-d36aae6143ed None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.870517 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72a728-afd8-42e1-a67d-d36aae6143ed None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.870754 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72a728-afd8-42e1-a67d-d36aae6143ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.872247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72a728-afd8-42e1-a67d-d36aae6143ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:49.873677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:49.874930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72a728-afd8-42e1-a67d-d36aae6143ed None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:49.877304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 17/34] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:49:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:49.879184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f1cb2-1c5b-490d-ae82-ac2e989e3b28 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.879184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f1cb2-1c5b-490d-ae82-ac2e989e3b28 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.879440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f1cb2-1c5b-490d-ae82-ac2e989e3b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.883394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125da34-1286-4a50-9320-19d2200f7b69 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:49.904400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-919f1cb2-1c5b-490d-ae82-ac2e989e3b28 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:49.906066 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-919f1cb2-1c5b-490d-ae82-ac2e989e3b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:49.930696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf794-7938-4cbd-9771-696134d6498b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:49.931372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf794-7938-4cbd-9771-696134d6498b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.931763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf794-7938-4cbd-9771-696134d6498b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.932054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf794-7938-4cbd-9771-696134d6498b None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.933784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdf794-7938-4cbd-9771-696134d6498b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:49:49.933784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdf794-7938-4cbd-9771-696134d6498b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:49.935317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdf794-7938-4cbd-9771-696134d6498b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:49.948749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 18/35] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:49.949589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf794-7938-4cbd-9771-696134d6498b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:49.987456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 18/36] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:49:49.989199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c47c-28d9-4778-9561-c021b6f5e73b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:49.989734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c47c-28d9-4778-9561-c021b6f5e73b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:49.990222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c47c-28d9-4778-9561-c021b6f5e73b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:49.991722 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 19/37] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:49.992685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580c47c-28d9-4778-9561-c021b6f5e73b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:50.054156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:50.057291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf794-7938-4cbd-9771-696134d6498b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:50.094594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7bc69-ea88-45e5-a0f0-78645892d105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:50.095483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7bc69-ea88-45e5-a0f0-78645892d105 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:50.096214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7bc69-ea88-45e5-a0f0-78645892d105 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:50.096509 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7bc69-ea88-45e5-a0f0-78645892d105 None admin] PATH_INFO: `/v3/domains/default/users/45ceaef67f814339b45f2d297054bedf/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:50.099235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7bc69-ea88-45e5-a0f0-78645892d105 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=45ceaef67f814339b45f2d297054bedf, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:50.101336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7bc69-ea88-45e5-a0f0-78645892d105 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:50.110640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580c47c-28d9-4778-9561-c021b6f5e73b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:50.143753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b4d26-c8c8-46af-92f6-5cc732eb4cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:50.144412 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b4d26-c8c8-46af-92f6-5cc732eb4cea None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:50.144694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b4d26-c8c8-46af-92f6-5cc732eb4cea None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:50.144997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b4d26-c8c8-46af-92f6-5cc732eb4cea None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:50.145710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b4d26-c8c8-46af-92f6-5cc732eb4cea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:50.147753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b4d26-c8c8-46af-92f6-5cc732eb4cea None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:50.196920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 19/38] 10.222.0.45 () {58 vars in 1361 bytes} [Tue Jul 5 15:49:49 2022] PUT /identity/v3/domains/default/users/45ceaef67f814339b45f2d297054bedf/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:49:50.198195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f1ba-2cf4-4033-a6e3-d964a9a1c330 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:50.198492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f1ba-2cf4-4033-a6e3-d964a9a1c330 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:50.198774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f1ba-2cf4-4033-a6e3-d964a9a1c330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:50.216971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-ff87f1ba-2cf4-4033-a6e3-d964a9a1c330 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:50.218570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ff87f1ba-2cf4-4033-a6e3-d964a9a1c330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:50.232191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:50.297528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 20/39] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:49:50.298169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87f1ba-2cf4-4033-a6e3-d964a9a1c330 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:50.418365 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87f1ba-2cf4-4033-a6e3-d964a9a1c330 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:50.457386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1e517-e925-4a26-8742-89f4ac892479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:50.458072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1e517-e925-4a26-8742-89f4ac892479 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:50.458362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1e517-e925-4a26-8742-89f4ac892479 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:50.458645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1e517-e925-4a26-8742-89f4ac892479 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:50.460059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1e517-e925-4a26-8742-89f4ac892479 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:50.461466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1e517-e925-4a26-8742-89f4ac892479 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:50.487800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 20/40] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:49:49 2022] POST /identity/v3/projects => generated 301 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:49:50.489426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fca41f-def1-4d81-bca8-3d7d3faa01bc None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:50.489717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fca41f-def1-4d81-bca8-3d7d3faa01bc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:50.489999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fca41f-def1-4d81-bca8-3d7d3faa01bc None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:50.491455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 21/41] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:50.492052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fca41f-def1-4d81-bca8-3d7d3faa01bc None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:50.553055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 21/42] 10.222.0.45 () {60 vars in 1131 bytes} [Tue Jul 5 15:49:50 2022] POST /identity/v3/roles => generated 225 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:49:50.554556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84969951-0dc2-429f-bde9-6a54b78676b9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:50.554851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84969951-0dc2-429f-bde9-6a54b78676b9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:50.555180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84969951-0dc2-429f-bde9-6a54b78676b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:50.571267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:50.578437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-84969951-0dc2-429f-bde9-6a54b78676b9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:50.580504 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-84969951-0dc2-429f-bde9-6a54b78676b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:50.624366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fca41f-def1-4d81-bca8-3d7d3faa01bc None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:50.634831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:50.670254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520320e-4421-40d0-b7a6-efcd2f2ee78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:50.671055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520320e-4421-40d0-b7a6-efcd2f2ee78e None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:50.671505 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520320e-4421-40d0-b7a6-efcd2f2ee78e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:50.671849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520320e-4421-40d0-b7a6-efcd2f2ee78e None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:50.672754 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520320e-4421-40d0-b7a6-efcd2f2ee78e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:50.674423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520320e-4421-40d0-b7a6-efcd2f2ee78e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:50.675875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 22/43] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:49:50.701226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84969951-0dc2-429f-bde9-6a54b78676b9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:50.789476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 22/44] 10.222.0.45 () {60 vars in 1131 bytes} [Tue Jul 5 15:49:50 2022] POST /identity/v3/roles => generated 223 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:49:50.813240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84969951-0dc2-429f-bde9-6a54b78676b9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:50.856848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00502802-1057-4638-9ff3-8c4052c6e1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:50.857719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00502802-1057-4638-9ff3-8c4052c6e1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:50.858012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00502802-1057-4638-9ff3-8c4052c6e1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:50.858290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00502802-1057-4638-9ff3-8c4052c6e1d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:50.859521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00502802-1057-4638-9ff3-8c4052c6e1d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:50.861347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00502802-1057-4638-9ff3-8c4052c6e1d6 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:50.863058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 23/45] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:49:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:49:50.869685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520320e-4421-40d0-b7a6-efcd2f2ee78e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:50.986980 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520320e-4421-40d0-b7a6-efcd2f2ee78e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:51.023564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2192126-e575-4e4c-bfe5-44271eb99201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:51.024232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2192126-e575-4e4c-bfe5-44271eb99201 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:51.024525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2192126-e575-4e4c-bfe5-44271eb99201 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:51.024836 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2192126-e575-4e4c-bfe5-44271eb99201 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:51.025570 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2192126-e575-4e4c-bfe5-44271eb99201 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:51.027047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2192126-e575-4e4c-bfe5-44271eb99201 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:51.116790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 23/46] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:49:50 2022] POST /identity/v3/projects => generated 312 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:49:52.249780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4bdf3a-adde-4c7b-bb7d-b15fba7b51ee None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:52.250294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4bdf3a-adde-4c7b-bb7d-b15fba7b51ee None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:52.250373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4bdf3a-adde-4c7b-bb7d-b15fba7b51ee None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:52.251721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 24/47] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:52.257554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9a414-35f8-469b-b038-84e67a9f47a6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:52.257983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9a414-35f8-469b-b038-84e67a9f47a6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:52.258128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9a414-35f8-469b-b038-84e67a9f47a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:52.277344 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2ff9a414-35f8-469b-b038-84e67a9f47a6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:52.278882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2ff9a414-35f8-469b-b038-84e67a9f47a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:52.292694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:52.345403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 24/48] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:49:52.369071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4bdf3a-adde-4c7b-bb7d-b15fba7b51ee None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:52.373074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:52.472621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4bdf3a-adde-4c7b-bb7d-b15fba7b51ee None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:52.504989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3b689-6041-4a4a-9dae-1601e87b8b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:52.505715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3b689-6041-4a4a-9dae-1601e87b8b4b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:52.506048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3b689-6041-4a4a-9dae-1601e87b8b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:52.506872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3b689-6041-4a4a-9dae-1601e87b8b4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:52.516745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3b689-6041-4a4a-9dae-1601e87b8b4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-33a3b689-6041-4a4a-9dae-1601e87b8b4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:49:52.524346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:49:52.525170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:49:52.526111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 25/49] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:49:52.531354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9a414-35f8-469b-b038-84e67a9f47a6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:52.628290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9a414-35f8-469b-b038-84e67a9f47a6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:52.660977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374d1b7-a039-4c6c-8ff6-dad0df67c006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:52.661647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374d1b7-a039-4c6c-8ff6-dad0df67c006 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:52.661936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374d1b7-a039-4c6c-8ff6-dad0df67c006 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:52.662711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374d1b7-a039-4c6c-8ff6-dad0df67c006 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:52.663471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374d1b7-a039-4c6c-8ff6-dad0df67c006 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:49:52.663784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374d1b7-a039-4c6c-8ff6-dad0df67c006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:52.665205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374d1b7-a039-4c6c-8ff6-dad0df67c006 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:52.673236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 25/50] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:52.679302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3b689-6041-4a4a-9dae-1601e87b8b4b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:52.844451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:52.882177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3b689-6041-4a4a-9dae-1601e87b8b4b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:52.916308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf6b55-fb9c-4398-99a1-3a98e2d58fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:52.917221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf6b55-fb9c-4398-99a1-3a98e2d58fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:52.917576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf6b55-fb9c-4398-99a1-3a98e2d58fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:52.917861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf6b55-fb9c-4398-99a1-3a98e2d58fa7 None admin] PATH_INFO: `/v3/users/45ceaef67f814339b45f2d297054bedf` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:52.919055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf6b55-fb9c-4398-99a1-3a98e2d58fa7 None admin] RBAC: Authorizing `identity:get_user(user_id=45ceaef67f814339b45f2d297054bedf)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:52.920506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf6b55-fb9c-4398-99a1-3a98e2d58fa7 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:52.922205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 26/51] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:49:52 2022] GET /identity/v3/users/45ceaef67f814339b45f2d297054bedf => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:52.927639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374d1b7-a039-4c6c-8ff6-dad0df67c006 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:52.940914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:53.026860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374d1b7-a039-4c6c-8ff6-dad0df67c006 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:53.059734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51774e0-34eb-4279-ad77-86ad72ae2a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:53.060535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51774e0-34eb-4279-ad77-86ad72ae2a05 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:53.060848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51774e0-34eb-4279-ad77-86ad72ae2a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:53.061128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51774e0-34eb-4279-ad77-86ad72ae2a05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:53.062301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51774e0-34eb-4279-ad77-86ad72ae2a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:53.063753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51774e0-34eb-4279-ad77-86ad72ae2a05 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:53.065403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 26/52] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:49:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:53.071044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf6b55-fb9c-4398-99a1-3a98e2d58fa7 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:53.125637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc598183-3fa7-4379-89cc-04c07d2af598 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:53.125865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc598183-3fa7-4379-89cc-04c07d2af598 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:53.126142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc598183-3fa7-4379-89cc-04c07d2af598 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:53.127575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 27/53] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:49:53.132404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca3316-8d14-4c48-89fb-c1e6678fc19a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:53.132702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca3316-8d14-4c48-89fb-c1e6678fc19a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:53.133020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca3316-8d14-4c48-89fb-c1e6678fc19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:53.149690 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c8ca3316-8d14-4c48-89fb-c1e6678fc19a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:53.150905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c8ca3316-8d14-4c48-89fb-c1e6678fc19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:53.170462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf6b55-fb9c-4398-99a1-3a98e2d58fa7 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:53.198758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:53.203519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e447b-6eb9-45bd-9284-f5d32de34183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:53.204176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e447b-6eb9-45bd-9284-f5d32de34183 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:53.209747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e447b-6eb9-45bd-9284-f5d32de34183 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:53.209747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e447b-6eb9-45bd-9284-f5d32de34183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:53.209747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e447b-6eb9-45bd-9284-f5d32de34183 None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.domain.id=default, user.id=45ceaef67f814339b45f2d297054bedf) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:49:53.209747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e447b-6eb9-45bd-9284-f5d32de34183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:53.209747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e447b-6eb9-45bd-9284-f5d32de34183 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:53.215729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 27/54] 10.222.0.45 () {56 vars in 1322 bytes} [Tue Jul 5 15:49:53 2022] GET /identity/v3/role_assignments?user.id=45ceaef67f814339b45f2d297054bedf&scope.domain.id=default&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:53.218187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 28/55] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:49:53.242214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e447b-6eb9-45bd-9284-f5d32de34183 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:53.351888 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e447b-6eb9-45bd-9284-f5d32de34183 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:53.385713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7574c-6935-4b6e-9abb-8d66520df8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:53.386430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7574c-6935-4b6e-9abb-8d66520df8dd None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:53.386720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7574c-6935-4b6e-9abb-8d66520df8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:53.386996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7574c-6935-4b6e-9abb-8d66520df8dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:53.388180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7574c-6935-4b6e-9abb-8d66520df8dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:53.389665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7574c-6935-4b6e-9abb-8d66520df8dd None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:53.391275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 28/56] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:49:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:53.397374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca3316-8d14-4c48-89fb-c1e6678fc19a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:53.408202 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:53.495881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca3316-8d14-4c48-89fb-c1e6678fc19a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:53.529244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55aa8a5-824a-444c-b914-57ea64a95c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:53.529905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55aa8a5-824a-444c-b914-57ea64a95c4f None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:53.530197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55aa8a5-824a-444c-b914-57ea64a95c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:53.530482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55aa8a5-824a-444c-b914-57ea64a95c4f None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:53.531206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55aa8a5-824a-444c-b914-57ea64a95c4f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:53.532687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55aa8a5-824a-444c-b914-57ea64a95c4f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:53.620075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 29/57] 10.222.0.45 () {60 vars in 1140 bytes} [Tue Jul 5 15:49:53 2022] POST /identity/v3/projects => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:49:55.629144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f08cc4-2ca5-4f01-b416-1bcb309a018b None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:55.629626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f08cc4-2ca5-4f01-b416-1bcb309a018b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:55.629626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f08cc4-2ca5-4f01-b416-1bcb309a018b None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:55.631525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 29/58] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:55.636523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a76b9-403e-4423-831b-490d03c41be4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:55.636903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a76b9-403e-4423-831b-490d03c41be4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:55.637114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a76b9-403e-4423-831b-490d03c41be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:55.654088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6f2a76b9-403e-4423-831b-490d03c41be4 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:55.655279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6f2a76b9-403e-4423-831b-490d03c41be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:55.708617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:55.719874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 30/59] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:49:55.743627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f08cc4-2ca5-4f01-b416-1bcb309a018b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:55.806726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:55.845215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f08cc4-2ca5-4f01-b416-1bcb309a018b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:55.877692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86802c-11c9-42cb-91c2-22053c7b0ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:55.878472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86802c-11c9-42cb-91c2-22053c7b0ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:55.878757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86802c-11c9-42cb-91c2-22053c7b0ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:55.879035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86802c-11c9-42cb-91c2-22053c7b0ec8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:55.880247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86802c-11c9-42cb-91c2-22053c7b0ec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:55.882192 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86802c-11c9-42cb-91c2-22053c7b0ec8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:55.883804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 30/60] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:49:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:49:55.889317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a76b9-403e-4423-831b-490d03c41be4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:55.985328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a76b9-403e-4423-831b-490d03c41be4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:56.018683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4df56-7090-439d-b39e-1de718944c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:56.019342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4df56-7090-439d-b39e-1de718944c52 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:56.019631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4df56-7090-439d-b39e-1de718944c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:56.019911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4df56-7090-439d-b39e-1de718944c52 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:56.020625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4df56-7090-439d-b39e-1de718944c52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:49:56.022142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4df56-7090-439d-b39e-1de718944c52 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:49:56.143006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 31/61] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:49:55 2022] POST /identity/v3/users => generated 275 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:49:59.469278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081a990-0d12-45f8-95d0-3da76110e28a None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.470283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081a990-0d12-45f8-95d0-3da76110e28a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.470774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081a990-0d12-45f8-95d0-3da76110e28a None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:59.472354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 31/62] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:59.478520 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca807e-172f-459c-ba78-e25b85031091 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.479460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca807e-172f-459c-ba78-e25b85031091 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.481048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a279cbf-d45f-460e-bd53-6b4e420008f3 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.481540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a279cbf-d45f-460e-bd53-6b4e420008f3 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.482016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a279cbf-d45f-460e-bd53-6b4e420008f3 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:59.483565 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 32/63] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:49:59.484012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca807e-172f-459c-ba78-e25b85031091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:59.488515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f35ab-a0d7-447a-b9e2-da9a9c0d58d0 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.488840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f35ab-a0d7-447a-b9e2-da9a9c0d58d0 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.489307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f35ab-a0d7-447a-b9e2-da9a9c0d58d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:59.515891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8eca807e-172f-459c-ba78-e25b85031091 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:59.516425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c70f35ab-a0d7-447a-b9e2-da9a9c0d58d0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:59.517522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8eca807e-172f-459c-ba78-e25b85031091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:59.518212 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c70f35ab-a0d7-447a-b9e2-da9a9c0d58d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:59.571672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:59.600102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 32/64] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:49:59.608785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbe0c9-5c40-47ce-877a-8700ab36fa5d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.609327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbe0c9-5c40-47ce-877a-8700ab36fa5d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.609735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 33/65] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:49:59.609868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbe0c9-5c40-47ce-877a-8700ab36fa5d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:59.611626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 33/66] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:59 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:59.628379 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f35ab-a0d7-447a-b9e2-da9a9c0d58d0 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:59.632535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d323d6-7e4f-4489-bd19-9935ad59d722 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.633093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d323d6-7e4f-4489-bd19-9935ad59d722 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.633535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d323d6-7e4f-4489-bd19-9935ad59d722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:59.654750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-65d323d6-7e4f-4489-bd19-9935ad59d722 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:49:59.656042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-65d323d6-7e4f-4489-bd19-9935ad59d722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:49:59.705377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:59.741373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 34/67] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:49:59.742229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d323d6-7e4f-4489-bd19-9935ad59d722 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:59.755119 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f35ab-a0d7-447a-b9e2-da9a9c0d58d0 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:59.794046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886422c-50a2-4e36-9c93-36dd4528b538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:59.794845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886422c-50a2-4e36-9c93-36dd4528b538 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.795136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886422c-50a2-4e36-9c93-36dd4528b538 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.795420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886422c-50a2-4e36-9c93-36dd4528b538 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:59.805874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886422c-50a2-4e36-9c93-36dd4528b538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-d886422c-50a2-4e36-9c93-36dd4528b538 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:49:59.813627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:49:59.814502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:49:59.815256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 34/68] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:49:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:49:59.816420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e111e-ea95-43bd-8550-16c71d1c59ca None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.816420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e111e-ea95-43bd-8550-16c71d1c59ca None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.816566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e111e-ea95-43bd-8550-16c71d1c59ca None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:59.818100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 35/69] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:49:59.823130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e111e-ea95-43bd-8550-16c71d1c59ca None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:59.858613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d323d6-7e4f-4489-bd19-9935ad59d722 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:59.866810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:59.894192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bdd36-221d-4695-bb75-1cf5a9bcf801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:59.894990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bdd36-221d-4695-bb75-1cf5a9bcf801 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.895281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bdd36-221d-4695-bb75-1cf5a9bcf801 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.895569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bdd36-221d-4695-bb75-1cf5a9bcf801 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:59.908637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bdd36-221d-4695-bb75-1cf5a9bcf801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-665bdd36-221d-4695-bb75-1cf5a9bcf801 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:49:59.917592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:49:59.918419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:49:59.919154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 35/70] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:49:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:49:59.919769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bdd36-221d-4695-bb75-1cf5a9bcf801 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:49:59.936608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e111e-ea95-43bd-8550-16c71d1c59ca None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:49:59.959366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:49:59.970629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59304f36-75ff-48e5-990a-69fd6183ed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:49:59.971435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304f36-75ff-48e5-990a-69fd6183ed58 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.971725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304f36-75ff-48e5-990a-69fd6183ed58 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.972005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304f36-75ff-48e5-990a-69fd6183ed58 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:49:59.982753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59304f36-75ff-48e5-990a-69fd6183ed58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-59304f36-75ff-48e5-990a-69fd6183ed58 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:49:59.990816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:49:59.991631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:49:59.992348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 36/71] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:49:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:49:59.993407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109e48d-fe25-461f-8f7c-393877a5d2d5 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:49:59.993470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109e48d-fe25-461f-8f7c-393877a5d2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:49:59.993814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109e48d-fe25-461f-8f7c-393877a5d2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.009460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c109e48d-fe25-461f-8f7c-393877a5d2d5 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:00.010656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c109e48d-fe25-461f-8f7c-393877a5d2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:00.026616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bdd36-221d-4695-bb75-1cf5a9bcf801 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.060281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd510f-e8ba-48d9-b368-9fe7e90fdc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.060999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd510f-e8ba-48d9-b368-9fe7e90fdc5f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.061291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd510f-e8ba-48d9-b368-9fe7e90fdc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.061577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd510f-e8ba-48d9-b368-9fe7e90fdc5f None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.062336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd510f-e8ba-48d9-b368-9fe7e90fdc5f None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:00.062650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd510f-e8ba-48d9-b368-9fe7e90fdc5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:00.064091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd510f-e8ba-48d9-b368-9fe7e90fdc5f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:00.072036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 36/72] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:49:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:00.072858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd510f-e8ba-48d9-b368-9fe7e90fdc5f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.075984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 37/73] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:00.082551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621d31f-7aa6-48d1-bb64-9692eaa167b2 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.082848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621d31f-7aa6-48d1-bb64-9692eaa167b2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.083126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621d31f-7aa6-48d1-bb64-9692eaa167b2 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.085487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 38/74] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:00.089905 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621d31f-7aa6-48d1-bb64-9692eaa167b2 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.103440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:00.174090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd510f-e8ba-48d9-b368-9fe7e90fdc5f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.190353 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621d31f-7aa6-48d1-bb64-9692eaa167b2 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.204070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:00.207076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dec9f-2634-4389-bf6e-1d613189637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.207750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dec9f-2634-4389-bf6e-1d613189637a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.208076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dec9f-2634-4389-bf6e-1d613189637a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.208324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dec9f-2634-4389-bf6e-1d613189637a None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.209733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dec9f-2634-4389-bf6e-1d613189637a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:00.219403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dec9f-2634-4389-bf6e-1d613189637a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:00.219403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dec9f-2634-4389-bf6e-1d613189637a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:00.219403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 37/75] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:00.220215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dec9f-2634-4389-bf6e-1d613189637a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.226316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7db03-c033-4fc5-8b20-3f737df2d204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.227242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7db03-c033-4fc5-8b20-3f737df2d204 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.227527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7db03-c033-4fc5-8b20-3f737df2d204 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.227802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7db03-c033-4fc5-8b20-3f737df2d204 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.228554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7db03-c033-4fc5-8b20-3f737df2d204 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:00.228886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7db03-c033-4fc5-8b20-3f737df2d204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:00.230672 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7db03-c033-4fc5-8b20-3f737df2d204 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:00.242749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 39/76] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:00.243929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f357e-2597-4d51-ae8e-e0c0005ad6bb None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.244214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f357e-2597-4d51-ae8e-e0c0005ad6bb None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.244496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f357e-2597-4d51-ae8e-e0c0005ad6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.260576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-bb7f357e-2597-4d51-ae8e-e0c0005ad6bb None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:00.261812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-bb7f357e-2597-4d51-ae8e-e0c0005ad6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:00.313738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:00.323573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dec9f-2634-4389-bf6e-1d613189637a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.327639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 40/77] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:00.328426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f357e-2597-4d51-ae8e-e0c0005ad6bb None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.361396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ce1a9-0bfb-4723-8885-1bc8de58456c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.362270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ce1a9-0bfb-4723-8885-1bc8de58456c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.362556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ce1a9-0bfb-4723-8885-1bc8de58456c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.362828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ce1a9-0bfb-4723-8885-1bc8de58456c None admin] PATH_INFO: `/v3/users/e64e34286da842d2abb6f973aee3a086` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.375826 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ce1a9-0bfb-4723-8885-1bc8de58456c None admin] RBAC: Authorizing `identity:get_user(user_id=e64e34286da842d2abb6f973aee3a086)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:00.377323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ce1a9-0bfb-4723-8885-1bc8de58456c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:00.379116 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 38/78] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/users/e64e34286da842d2abb6f973aee3a086 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:00.379736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ce1a9-0bfb-4723-8885-1bc8de58456c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.432610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f357e-2597-4d51-ae8e-e0c0005ad6bb None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.434908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:00.465911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fb0b1-0238-4c77-a5ba-ee6f697e45cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.466699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fb0b1-0238-4c77-a5ba-ee6f697e45cc None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.466984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fb0b1-0238-4c77-a5ba-ee6f697e45cc None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.467263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fb0b1-0238-4c77-a5ba-ee6f697e45cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.477904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fb0b1-0238-4c77-a5ba-ee6f697e45cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:00.479126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ce1a9-0bfb-4723-8885-1bc8de58456c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-b03fb0b1-0238-4c77-a5ba-ee6f697e45cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:00.485531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:00.486352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:00.487253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 41/79] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:00.488528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fb0b1-0238-4c77-a5ba-ee6f697e45cc None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.515788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479a0a8-6462-4240-818e-b7f6b2b6434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.516661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479a0a8-6462-4240-818e-b7f6b2b6434e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.516980 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479a0a8-6462-4240-818e-b7f6b2b6434e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.517264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f479a0a8-6462-4240-818e-b7f6b2b6434e None admin] PATH_INFO: `/v3/users/45ceaef67f814339b45f2d297054bedf` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.518452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479a0a8-6462-4240-818e-b7f6b2b6434e None admin] RBAC: Authorizing `identity:get_user(user_id=45ceaef67f814339b45f2d297054bedf)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:00.519904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f479a0a8-6462-4240-818e-b7f6b2b6434e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:00.521604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 39/80] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/users/45ceaef67f814339b45f2d297054bedf => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:00.522362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479a0a8-6462-4240-818e-b7f6b2b6434e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.539906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:00.590871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fb0b1-0238-4c77-a5ba-ee6f697e45cc None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.621206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f479a0a8-6462-4240-818e-b7f6b2b6434e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.623038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5971d-0f92-41a3-a33e-e3d1deba9b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.623901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5971d-0f92-41a3-a33e-e3d1deba9b42 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.624187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5971d-0f92-41a3-a33e-e3d1deba9b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.624460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5971d-0f92-41a3-a33e-e3d1deba9b42 None admin] PATH_INFO: `/v3/users/e64e34286da842d2abb6f973aee3a086` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.625690 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5971d-0f92-41a3-a33e-e3d1deba9b42 None admin] RBAC: Authorizing `identity:get_user(user_id=e64e34286da842d2abb6f973aee3a086)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:00.627253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5971d-0f92-41a3-a33e-e3d1deba9b42 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:00.628903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 42/81] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/users/e64e34286da842d2abb6f973aee3a086 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:00.629656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5971d-0f92-41a3-a33e-e3d1deba9b42 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.654188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d91ab9-38a0-4c5d-a443-cc52e0e3839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.654972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d91ab9-38a0-4c5d-a443-cc52e0e3839b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.655252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d91ab9-38a0-4c5d-a443-cc52e0e3839b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.655523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d91ab9-38a0-4c5d-a443-cc52e0e3839b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.656691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d91ab9-38a0-4c5d-a443-cc52e0e3839b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:00.658199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d91ab9-38a0-4c5d-a443-cc52e0e3839b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:00.659791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 40/82] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:00.664098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:00.664862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d91ab9-38a0-4c5d-a443-cc52e0e3839b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.733013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5971d-0f92-41a3-a33e-e3d1deba9b42 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.747402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:00.763529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d91ab9-38a0-4c5d-a443-cc52e0e3839b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.766316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3ffde-7b2b-442b-8983-0b355cc0edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.766955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3ffde-7b2b-442b-8983-0b355cc0edb3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.767240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3ffde-7b2b-442b-8983-0b355cc0edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.767516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3ffde-7b2b-442b-8983-0b355cc0edb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.796616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52284b10-7c7c-400d-b944-afdde98740ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.797301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52284b10-7c7c-400d-b944-afdde98740ce None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.797591 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52284b10-7c7c-400d-b944-afdde98740ce None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.797874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52284b10-7c7c-400d-b944-afdde98740ce None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.798623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52284b10-7c7c-400d-b944-afdde98740ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:00.798934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52284b10-7c7c-400d-b944-afdde98740ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:00.800331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52284b10-7c7c-400d-b944-afdde98740ce None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:00.809207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 41/83] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:00.809898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52284b10-7c7c-400d-b944-afdde98740ce None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.838007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3ffde-7b2b-442b-8983-0b355cc0edb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:00.839479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3ffde-7b2b-442b-8983-0b355cc0edb3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:00.869471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:00.909594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52284b10-7c7c-400d-b944-afdde98740ce None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:00.937106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 43/84] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:00.937934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3ffde-7b2b-442b-8983-0b355cc0edb3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:00.947080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74e489-8676-44fa-9911-496625f04d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:00.947733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74e489-8676-44fa-9911-496625f04d2c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:00.948031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74e489-8676-44fa-9911-496625f04d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:00.948315 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74e489-8676-44fa-9911-496625f04d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:00.963856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:01.016055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74e489-8676-44fa-9911-496625f04d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:01.017553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74e489-8676-44fa-9911-496625f04d2c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:01.033163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3ffde-7b2b-442b-8983-0b355cc0edb3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:01.061562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:01.066718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb181ee4-4c16-4474-ba7d-a679110dd6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:01.067357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb181ee4-4c16-4474-ba7d-a679110dd6da None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:01.067646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb181ee4-4c16-4474-ba7d-a679110dd6da None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:01.067923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb181ee4-4c16-4474-ba7d-a679110dd6da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:01.113131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 42/85] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:01.113936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74e489-8676-44fa-9911-496625f04d2c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:01.136156 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb181ee4-4c16-4474-ba7d-a679110dd6da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:01.137585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb181ee4-4c16-4474-ba7d-a679110dd6da None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:01.159077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:01.212270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74e489-8676-44fa-9911-496625f04d2c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:01.233847 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 44/86] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:01.234673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb181ee4-4c16-4474-ba7d-a679110dd6da None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:01.247728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103b9d-afff-4221-ba0b-0a627c679352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:01.248395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103b9d-afff-4221-ba0b-0a627c679352 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:01.248681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103b9d-afff-4221-ba0b-0a627c679352 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:01.248991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103b9d-afff-4221-ba0b-0a627c679352 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:01.249717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f103b9d-afff-4221-ba0b-0a627c679352 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:01.251160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f103b9d-afff-4221-ba0b-0a627c679352 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:01.266173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:01.331634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 43/87] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:01 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:01.331906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb181ee4-4c16-4474-ba7d-a679110dd6da None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:01.332384 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103b9d-afff-4221-ba0b-0a627c679352 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:01.373471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0db4a-34a9-4bd1-b36a-8e77f1392c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:01.374355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0db4a-34a9-4bd1-b36a-8e77f1392c0b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:01.374641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0db4a-34a9-4bd1-b36a-8e77f1392c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:01.374917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0db4a-34a9-4bd1-b36a-8e77f1392c0b None admin] PATH_INFO: `/v3/users/e64e34286da842d2abb6f973aee3a086` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:01.376216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0db4a-34a9-4bd1-b36a-8e77f1392c0b None admin] RBAC: Authorizing `identity:get_user(user_id=e64e34286da842d2abb6f973aee3a086)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:01.377875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0db4a-34a9-4bd1-b36a-8e77f1392c0b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:01.379752 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 45/88] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:01 2022] GET /identity/v3/users/e64e34286da842d2abb6f973aee3a086 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:01.380612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0db4a-34a9-4bd1-b36a-8e77f1392c0b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:01.446678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103b9d-afff-4221-ba0b-0a627c679352 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:01.448512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:01.489294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a64c77-a76a-478f-9d6c-94bf34e01e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:01.490195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a64c77-a76a-478f-9d6c-94bf34e01e6f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:01.490483 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a64c77-a76a-478f-9d6c-94bf34e01e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:01.490931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a64c77-a76a-478f-9d6c-94bf34e01e6f None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:01.492681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a64c77-a76a-478f-9d6c-94bf34e01e6f None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:01.493963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a64c77-a76a-478f-9d6c-94bf34e01e6f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:01.496473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 44/89] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:01 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:01.496473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a64c77-a76a-478f-9d6c-94bf34e01e6f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:01.498248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:01.503390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0db4a-34a9-4bd1-b36a-8e77f1392c0b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:01.548866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b5c09-9d83-45c9-a74e-519c31e4a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:01.549745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b5c09-9d83-45c9-a74e-519c31e4a943 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:01.550037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b5c09-9d83-45c9-a74e-519c31e4a943 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:01.550323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b5c09-9d83-45c9-a74e-519c31e4a943 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:01.551527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b5c09-9d83-45c9-a74e-519c31e4a943 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:01.553050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b5c09-9d83-45c9-a74e-519c31e4a943 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:01.554954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 46/90] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:01 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:01.555537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b5c09-9d83-45c9-a74e-519c31e4a943 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:01.626987 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a64c77-a76a-478f-9d6c-94bf34e01e6f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:01.668119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347fc6d-acea-4ba6-9669-fdbc13634946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:01.669464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347fc6d-acea-4ba6-9669-fdbc13634946 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:01.669754 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347fc6d-acea-4ba6-9669-fdbc13634946 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:01.670040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347fc6d-acea-4ba6-9669-fdbc13634946 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:01.671373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347fc6d-acea-4ba6-9669-fdbc13634946 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:01.672859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347fc6d-acea-4ba6-9669-fdbc13634946 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:01.674574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 45/91] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:01 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:01.675330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347fc6d-acea-4ba6-9669-fdbc13634946 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:01.687170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b5c09-9d83-45c9-a74e-519c31e4a943 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:01.723642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:01.735819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbadefc-9b47-4a9f-b399-1aff6981676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:01.741932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbadefc-9b47-4a9f-b399-1aff6981676d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:01.741932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbadefc-9b47-4a9f-b399-1aff6981676d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:01.741932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbadefc-9b47-4a9f-b399-1aff6981676d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:01.787177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:01.787496 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347fc6d-acea-4ba6-9669-fdbc13634946 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:01.827829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f3515-fca3-4e57-9c21-14ce39dfec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:01.828490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f3515-fca3-4e57-9c21-14ce39dfec73 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:01.828796 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f3515-fca3-4e57-9c21-14ce39dfec73 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:01.829123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f3515-fca3-4e57-9c21-14ce39dfec73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:01.829968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f3515-fca3-4e57-9c21-14ce39dfec73 None admin] RBAC: Adding query filter params (role.id=61e62f29eafd45c8b9fb99205a60bc4e, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user.id=e64e34286da842d2abb6f973aee3a086) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:01.830365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f3515-fca3-4e57-9c21-14ce39dfec73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:01.831822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f3515-fca3-4e57-9c21-14ce39dfec73 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:01.833244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbadefc-9b47-4a9f-b399-1aff6981676d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:01.836039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbadefc-9b47-4a9f-b399-1aff6981676d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:01.838587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 46/92] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:01 2022] GET /identity/v3/role_assignments?user.id=e64e34286da842d2abb6f973aee3a086&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:01.839170 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f3515-fca3-4e57-9c21-14ce39dfec73 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:02.014619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 47/93] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:02.016080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbadefc-9b47-4a9f-b399-1aff6981676d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:02.036450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f3515-fca3-4e57-9c21-14ce39dfec73 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:02.049105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:02.111070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46b0b-1548-4369-9dc2-05d59a2df033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:02.111730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46b0b-1548-4369-9dc2-05d59a2df033 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:02.112020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46b0b-1548-4369-9dc2-05d59a2df033 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:02.112300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46b0b-1548-4369-9dc2-05d59a2df033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:02.113162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46b0b-1548-4369-9dc2-05d59a2df033 None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user.id=45ceaef67f814339b45f2d297054bedf) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:02.113478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46b0b-1548-4369-9dc2-05d59a2df033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:02.118696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46b0b-1548-4369-9dc2-05d59a2df033 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:02.128604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 47/94] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:01 2022] GET /identity/v3/role_assignments?user.id=45ceaef67f814339b45f2d297054bedf&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:02.132742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46b0b-1548-4369-9dc2-05d59a2df033 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:02.175034 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:02.264487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbadefc-9b47-4a9f-b399-1aff6981676d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:02.358611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46b0b-1548-4369-9dc2-05d59a2df033 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:02.393480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b371f-1943-455b-8323-49a9d07ca98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:02.394154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b371f-1943-455b-8323-49a9d07ca98f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:02.394442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b371f-1943-455b-8323-49a9d07ca98f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:02.394720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b371f-1943-455b-8323-49a9d07ca98f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:02.395569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b371f-1943-455b-8323-49a9d07ca98f None admin] RBAC: Adding query filter params (role.id=2991af796dcd45e2b86208339339b977, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user.id=e64e34286da842d2abb6f973aee3a086) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:02.395891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b371f-1943-455b-8323-49a9d07ca98f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:02.397520 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3d81d-67fe-49d0-8aa3-a40d030ae9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:02.397656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b371f-1943-455b-8323-49a9d07ca98f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:02.398713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3d81d-67fe-49d0-8aa3-a40d030ae9dd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:02.399197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3d81d-67fe-49d0-8aa3-a40d030ae9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:02.399928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3d81d-67fe-49d0-8aa3-a40d030ae9dd None admin] PATH_INFO: `/v3/projects/e715b2d9555946578a479c254f7ef8dd` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:02.401382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3d81d-67fe-49d0-8aa3-a40d030ae9dd None admin] RBAC: Authorizing `identity:get_project(project_id=e715b2d9555946578a479c254f7ef8dd)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:02.404008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3d81d-67fe-49d0-8aa3-a40d030ae9dd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:02.404963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 48/95] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:02 2022] GET /identity/v3/role_assignments?user.id=e64e34286da842d2abb6f973aee3a086&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=2991af796dcd45e2b86208339339b977 => generated 259 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:02.406209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 48/96] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:02 2022] GET /identity/v3/projects/e715b2d9555946578a479c254f7ef8dd => generated 312 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:02.412918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b371f-1943-455b-8323-49a9d07ca98f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:02.571603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b371f-1943-455b-8323-49a9d07ca98f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:02.612335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:02.618871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e1747-174a-438e-a719-59b8828367a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:02.619533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1747-174a-438e-a719-59b8828367a8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:02.619862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1747-174a-438e-a719-59b8828367a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:02.620141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1747-174a-438e-a719-59b8828367a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:02.631241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e1747-174a-438e-a719-59b8828367a8 None admin] RBAC: Adding query filter params (role.id=61e62f29eafd45c8b9fb99205a60bc4e, scope.project.id=e715b2d9555946578a479c254f7ef8dd, user.id=e64e34286da842d2abb6f973aee3a086) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:02.631241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e1747-174a-438e-a719-59b8828367a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:02.631241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e1747-174a-438e-a719-59b8828367a8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:02.633642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 49/97] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:02 2022] GET /identity/v3/role_assignments?user.id=e64e34286da842d2abb6f973aee3a086&scope.project.id=e715b2d9555946578a479c254f7ef8dd&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:03.941551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041d64d-22d3-4f11-8333-70ad58555f8f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:03.942163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041d64d-22d3-4f11-8333-70ad58555f8f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:03.942468 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041d64d-22d3-4f11-8333-70ad58555f8f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:03.944665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 49/98] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:03.959348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6e6ed-29cf-4d80-a92d-40ad1771d536 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:03.959859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6e6ed-29cf-4d80-a92d-40ad1771d536 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:03.959941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6e6ed-29cf-4d80-a92d-40ad1771d536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:03.981302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-55d6e6ed-29cf-4d80-a92d-40ad1771d536 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:03.982897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-55d6e6ed-29cf-4d80-a92d-40ad1771d536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:04.062600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 50/99] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:04.103053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041d64d-22d3-4f11-8333-70ad58555f8f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:04.149639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:04.198890 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3743ba-6f1d-4e7f-8be5-0fa0b8641dbc None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:04.199496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3743ba-6f1d-4e7f-8be5-0fa0b8641dbc None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:04.199989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3743ba-6f1d-4e7f-8be5-0fa0b8641dbc None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:04.201897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 51/100] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:04.206710 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1868638-dcbe-4175-85c1-64fcd969f368 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:04.207254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1868638-dcbe-4175-85c1-64fcd969f368 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:04.207736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1868638-dcbe-4175-85c1-64fcd969f368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:04.238529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b1868638-dcbe-4175-85c1-64fcd969f368 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:04.240093 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b1868638-dcbe-4175-85c1-64fcd969f368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:04.254018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041d64d-22d3-4f11-8333-70ad58555f8f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:04.358492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883c116-9715-4f39-83a8-3920eb853d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:04.359316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883c116-9715-4f39-83a8-3920eb853d88 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:04.359611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883c116-9715-4f39-83a8-3920eb853d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:04.359898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883c116-9715-4f39-83a8-3920eb853d88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:04.365677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883c116-9715-4f39-83a8-3920eb853d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:04.367190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883c116-9715-4f39-83a8-3920eb853d88 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:04.368952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 50/101] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:04.375367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883c116-9715-4f39-83a8-3920eb853d88 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:04.409208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 52/102] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:04.450292 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01152b-45a3-412a-bb03-f1c72fc510ae None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:04.450581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01152b-45a3-412a-bb03-f1c72fc510ae None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:04.450793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01152b-45a3-412a-bb03-f1c72fc510ae None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:04.452317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 53/103] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:50:04.452926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01152b-45a3-412a-bb03-f1c72fc510ae None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:04.526853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:04.577831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01152b-45a3-412a-bb03-f1c72fc510ae None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:04.603024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883c116-9715-4f39-83a8-3920eb853d88 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:04.605125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:04.617654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff31f5b-7a2a-432d-a8c7-54098d96fe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:04.620160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff31f5b-7a2a-432d-a8c7-54098d96fe9a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:04.620160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff31f5b-7a2a-432d-a8c7-54098d96fe9a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:04.620160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff31f5b-7a2a-432d-a8c7-54098d96fe9a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:04.632204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff31f5b-7a2a-432d-a8c7-54098d96fe9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-bff31f5b-7a2a-432d-a8c7-54098d96fe9a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:04.641863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:04.642765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:04.643533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 54/104] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:04.649310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0af9f-fab8-4cd9-87b2-a1b136ca1fbf None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:04.649818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0af9f-fab8-4cd9-87b2-a1b136ca1fbf None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:04.650214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0af9f-fab8-4cd9-87b2-a1b136ca1fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:04.670388 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-9ae0af9f-fab8-4cd9-87b2-a1b136ca1fbf None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:04.671727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9ae0af9f-fab8-4cd9-87b2-a1b136ca1fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:04.672081 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacabd5-d6c4-482e-a9bc-cedcbc109cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:04.672978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacabd5-d6c4-482e-a9bc-cedcbc109cbc None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:04.673483 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacabd5-d6c4-482e-a9bc-cedcbc109cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:04.673946 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacabd5-d6c4-482e-a9bc-cedcbc109cbc None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:04.674857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacabd5-d6c4-482e-a9bc-cedcbc109cbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:04.676506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacabd5-d6c4-482e-a9bc-cedcbc109cbc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:04.741687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 55/105] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:04.742750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0af9f-fab8-4cd9-87b2-a1b136ca1fbf None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:04.792237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 51/106] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:50:04 2022] POST /identity/v3/projects => generated 302 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:50:04.794201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacabd5-d6c4-482e-a9bc-cedcbc109cbc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:04.819479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:05.029773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0af9f-fab8-4cd9-87b2-a1b136ca1fbf None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:05.072244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b2470-fcad-4a28-9b7f-8f8a73199821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:05.072949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b2470-fcad-4a28-9b7f-8f8a73199821 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.073236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b2470-fcad-4a28-9b7f-8f8a73199821 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.073514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b2470-fcad-4a28-9b7f-8f8a73199821 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.074268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b2470-fcad-4a28-9b7f-8f8a73199821 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:05.074576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b2470-fcad-4a28-9b7f-8f8a73199821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:05.075975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b2470-fcad-4a28-9b7f-8f8a73199821 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:05.085773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 56/107] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:05.094378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b2470-fcad-4a28-9b7f-8f8a73199821 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:05.124054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacabd5-d6c4-482e-a9bc-cedcbc109cbc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:05.180409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fda8d-de8b-41c7-8693-5d3d23a06515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:05.181657 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fda8d-de8b-41c7-8693-5d3d23a06515 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.182217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fda8d-de8b-41c7-8693-5d3d23a06515 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.182499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fda8d-de8b-41c7-8693-5d3d23a06515 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.197949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fda8d-de8b-41c7-8693-5d3d23a06515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-046fda8d-de8b-41c7-8693-5d3d23a06515 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:05.207430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:05.208266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:05.209614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 52/108] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:05.212671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21072bb6-10da-41df-93e0-25fadaa2159d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.212961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21072bb6-10da-41df-93e0-25fadaa2159d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.216031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21072bb6-10da-41df-93e0-25fadaa2159d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.216031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 53/109] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:05.219617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21072bb6-10da-41df-93e0-25fadaa2159d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:05.234842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b2470-fcad-4a28-9b7f-8f8a73199821 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:05.244343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:05.257891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:05.273019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de524a-e3b5-48a6-b07d-dd3833f46e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:05.273019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de524a-e3b5-48a6-b07d-dd3833f46e6c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.273019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de524a-e3b5-48a6-b07d-dd3833f46e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.273019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de524a-e3b5-48a6-b07d-dd3833f46e6c None admin] PATH_INFO: `/v3/users/8c64f203641c4524840aa140fe420816` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.290776 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de524a-e3b5-48a6-b07d-dd3833f46e6c None admin] RBAC: Authorizing `identity:get_user(user_id=8c64f203641c4524840aa140fe420816)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:05.290776 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de524a-e3b5-48a6-b07d-dd3833f46e6c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:05.292085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 57/110] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:05 2022] GET /identity/v3/users/8c64f203641c4524840aa140fe420816 => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:05.294294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b10bd4-c54c-46ad-97b8-266e3817cf75 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.294619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b10bd4-c54c-46ad-97b8-266e3817cf75 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.294909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b10bd4-c54c-46ad-97b8-266e3817cf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.312660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-88b10bd4-c54c-46ad-97b8-266e3817cf75 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:05.313929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-88b10bd4-c54c-46ad-97b8-266e3817cf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:05.387029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 58/111] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:05.387947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b10bd4-c54c-46ad-97b8-266e3817cf75 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:05.488741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:05.494432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21072bb6-10da-41df-93e0-25fadaa2159d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:05.504988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b10bd4-c54c-46ad-97b8-266e3817cf75 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:05.537381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae552ddc-19a4-477d-add3-886841cb47d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:05.538188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae552ddc-19a4-477d-add3-886841cb47d8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.538575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae552ddc-19a4-477d-add3-886841cb47d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.538852 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae552ddc-19a4-477d-add3-886841cb47d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.539720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae552ddc-19a4-477d-add3-886841cb47d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:05.540036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae552ddc-19a4-477d-add3-886841cb47d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:05.540194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d430891-6201-45aa-9101-738ecf6f0df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:05.540942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d430891-6201-45aa-9101-738ecf6f0df5 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.541181 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d430891-6201-45aa-9101-738ecf6f0df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.541466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d430891-6201-45aa-9101-738ecf6f0df5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.542216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae552ddc-19a4-477d-add3-886841cb47d8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:05.552689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 54/112] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:05.557701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae552ddc-19a4-477d-add3-886841cb47d8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:05.621950 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d430891-6201-45aa-9101-738ecf6f0df5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:05.623583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d430891-6201-45aa-9101-738ecf6f0df5 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:05.662723 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:05.694360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae552ddc-19a4-477d-add3-886841cb47d8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:05.732028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:05.733480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20415684-4f41-49ba-bcc3-7415910de9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:05.734297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20415684-4f41-49ba-bcc3-7415910de9de None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.734590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20415684-4f41-49ba-bcc3-7415910de9de None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.734919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20415684-4f41-49ba-bcc3-7415910de9de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.741712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 59/113] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:05.745074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a2f7a-16a7-450a-96a6-2c6b832a7a12 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.745074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a2f7a-16a7-450a-96a6-2c6b832a7a12 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.745074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a2f7a-16a7-450a-96a6-2c6b832a7a12 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.746348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20415684-4f41-49ba-bcc3-7415910de9de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:05.750527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 60/114] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:05.751422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a2f7a-16a7-450a-96a6-2c6b832a7a12 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-20415684-4f41-49ba-bcc3-7415910de9de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:05.754970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:05.756400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:05.757302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 55/115] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:05.757909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20415684-4f41-49ba-bcc3-7415910de9de None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:05.871335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a2f7a-16a7-450a-96a6-2c6b832a7a12 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:05.878255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20415684-4f41-49ba-bcc3-7415910de9de None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:05.907666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ee095-0f30-4aa6-8dee-f1414f4ccf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:05.908855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ee095-0f30-4aa6-8dee-f1414f4ccf1f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.909150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ee095-0f30-4aa6-8dee-f1414f4ccf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.909428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ee095-0f30-4aa6-8dee-f1414f4ccf1f None admin] PATH_INFO: `/v3/users/e64e34286da842d2abb6f973aee3a086` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.910640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ee095-0f30-4aa6-8dee-f1414f4ccf1f None admin] RBAC: Authorizing `identity:get_user(user_id=e64e34286da842d2abb6f973aee3a086)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:05.912158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ee095-0f30-4aa6-8dee-f1414f4ccf1f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:05.913848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 61/116] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:05 2022] GET /identity/v3/users/e64e34286da842d2abb6f973aee3a086 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:05.914982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c261f04-888b-43fd-a287-9580ee676158 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.915268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c261f04-888b-43fd-a287-9580ee676158 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.915546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c261f04-888b-43fd-a287-9580ee676158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.920129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251860f-053c-4ae9-992c-e3c7734a2e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:05.921190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251860f-053c-4ae9-992c-e3c7734a2e67 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:05.921639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251860f-053c-4ae9-992c-e3c7734a2e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:05.922081 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251860f-053c-4ae9-992c-e3c7734a2e67 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:05.923441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251860f-053c-4ae9-992c-e3c7734a2e67 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:05.925090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251860f-053c-4ae9-992c-e3c7734a2e67 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:05.932476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 56/117] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:05 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:05.933481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251860f-053c-4ae9-992c-e3c7734a2e67 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:05.933869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-5c261f04-888b-43fd-a287-9580ee676158 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:05.935093 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5c261f04-888b-43fd-a287-9580ee676158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:05.939029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:05.996949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:06.005629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 62/118] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:06.007650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc003f-913b-4fe6-815d-3cc99cc82cf8 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.008097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc003f-913b-4fe6-815d-3cc99cc82cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.008524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc003f-913b-4fe6-815d-3cc99cc82cf8 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.009976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 63/119] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:06.010811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc003f-913b-4fe6-815d-3cc99cc82cf8 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:06.048169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251860f-053c-4ae9-992c-e3c7734a2e67 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:06.084851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf03a39-f374-4028-836a-55232a2152f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:06.085521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf03a39-f374-4028-836a-55232a2152f9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.085808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf03a39-f374-4028-836a-55232a2152f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.086085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf03a39-f374-4028-836a-55232a2152f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.086826 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf03a39-f374-4028-836a-55232a2152f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:06.087417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf03a39-f374-4028-836a-55232a2152f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:06.088528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf03a39-f374-4028-836a-55232a2152f9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:06.097634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 57/120] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:06.103402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf03a39-f374-4028-836a-55232a2152f9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:06.124187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc003f-913b-4fe6-815d-3cc99cc82cf8 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:06.162330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74af578-2136-43c3-853c-493f37f01220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:06.163189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74af578-2136-43c3-853c-493f37f01220 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.163473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74af578-2136-43c3-853c-493f37f01220 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.163748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74af578-2136-43c3-853c-493f37f01220 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.164953 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74af578-2136-43c3-853c-493f37f01220 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:06.166492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74af578-2136-43c3-853c-493f37f01220 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:06.168186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 64/121] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:06 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:06.169334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6646f-5457-4600-b619-a8fa28d2665e None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.169622 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6646f-5457-4600-b619-a8fa28d2665e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.169899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6646f-5457-4600-b619-a8fa28d2665e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.178319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:06.186709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-a3d6646f-5457-4600-b619-a8fa28d2665e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:06.187873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a3d6646f-5457-4600-b619-a8fa28d2665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:06.212479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf03a39-f374-4028-836a-55232a2152f9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:06.248303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:06.248721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d3fee-e6a0-4273-8572-4a3606c676cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:06.249780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d3fee-e6a0-4273-8572-4a3606c676cf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.250076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d3fee-e6a0-4273-8572-4a3606c676cf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.250356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d3fee-e6a0-4273-8572-4a3606c676cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.251204 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d3fee-e6a0-4273-8572-4a3606c676cf None admin] RBAC: Adding query filter params (role.id=583fe587d8d94b0fad411aeefb4181e0, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user.id=8c64f203641c4524840aa140fe420816) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:06.251531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d3fee-e6a0-4273-8572-4a3606c676cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:06.253022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d3fee-e6a0-4273-8572-4a3606c676cf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:06.259373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 58/122] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:06 2022] GET /identity/v3/role_assignments?user.id=8c64f203641c4524840aa140fe420816&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=583fe587d8d94b0fad411aeefb4181e0 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:06.259807 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 65/123] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:06.266190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6646f-5457-4600-b619-a8fa28d2665e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:06.273814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d3fee-e6a0-4273-8572-4a3606c676cf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:06.405469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d3fee-e6a0-4273-8572-4a3606c676cf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:06.410385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6646f-5457-4600-b619-a8fa28d2665e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:06.450682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:06.456604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9b593-0bdc-45fc-9a1a-2290e7b5bd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:06.457522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b593-0bdc-45fc-9a1a-2290e7b5bd4d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.457809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b593-0bdc-45fc-9a1a-2290e7b5bd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.458087 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b593-0bdc-45fc-9a1a-2290e7b5bd4d None admin] PATH_INFO: `/v3/users/45ceaef67f814339b45f2d297054bedf` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.460244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9b593-0bdc-45fc-9a1a-2290e7b5bd4d None admin] RBAC: Authorizing `identity:get_user(user_id=45ceaef67f814339b45f2d297054bedf)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:06.461733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9b593-0bdc-45fc-9a1a-2290e7b5bd4d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:06.463015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652c536-4433-4d68-951e-ecb66735ec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:06.472601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 66/124] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:06 2022] GET /identity/v3/users/45ceaef67f814339b45f2d297054bedf => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:06.472601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9b593-0bdc-45fc-9a1a-2290e7b5bd4d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:06.472920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652c536-4433-4d68-951e-ecb66735ec5d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.472920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652c536-4433-4d68-951e-ecb66735ec5d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.472920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652c536-4433-4d68-951e-ecb66735ec5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.483175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652c536-4433-4d68-951e-ecb66735ec5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4652c536-4433-4d68-951e-ecb66735ec5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:06.495427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:06.496251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:06.497035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:06.498428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 59/125] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:06.498866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652c536-4433-4d68-951e-ecb66735ec5d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:06.533830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:06.580027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9b593-0bdc-45fc-9a1a-2290e7b5bd4d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:06.629189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652c536-4433-4d68-951e-ecb66735ec5d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:06.630423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3bdcc-0fa3-4991-b83b-5b9b83b9c197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:06.632644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3bdcc-0fa3-4991-b83b-5b9b83b9c197 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.633142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3bdcc-0fa3-4991-b83b-5b9b83b9c197 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.633718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3bdcc-0fa3-4991-b83b-5b9b83b9c197 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/users/e64e34286da842d2abb6f973aee3a086/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.635946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3bdcc-0fa3-4991-b83b-5b9b83b9c197 None admin] RBAC: Authorizing `identity:create_grant(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user_id=e64e34286da842d2abb6f973aee3a086, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:06.638189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3bdcc-0fa3-4991-b83b-5b9b83b9c197 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:06.666228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e0f90-e3dc-4f6d-8880-823011dc535a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:06.667112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e0f90-e3dc-4f6d-8880-823011dc535a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.667473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e0f90-e3dc-4f6d-8880-823011dc535a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.667825 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e0f90-e3dc-4f6d-8880-823011dc535a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.678591 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e0f90-e3dc-4f6d-8880-823011dc535a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-075e0f90-e3dc-4f6d-8880-823011dc535a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:06.686244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:06.687092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:06.687906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:06.692325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 60/126] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:06.693290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e0f90-e3dc-4f6d-8880-823011dc535a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:06.705898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 67/127] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:50:06 2022] PUT /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/users/e64e34286da842d2abb6f973aee3a086/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:06.707012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3bdcc-0fa3-4991-b83b-5b9b83b9c197 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:06.789146 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:06.839043 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3bdcc-0fa3-4991-b83b-5b9b83b9c197 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:06.864739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:06.881886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27d9a2-f933-4eeb-8593-74a4bdd95d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:06.882658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27d9a2-f933-4eeb-8593-74a4bdd95d17 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.882931 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27d9a2-f933-4eeb-8593-74a4bdd95d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.883286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27d9a2-f933-4eeb-8593-74a4bdd95d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.884065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27d9a2-f933-4eeb-8593-74a4bdd95d17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:06.884432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27d9a2-f933-4eeb-8593-74a4bdd95d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:06.886447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27d9a2-f933-4eeb-8593-74a4bdd95d17 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:06.897861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 68/128] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:06.898770 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27d9a2-f933-4eeb-8593-74a4bdd95d17 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:06.943822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e0f90-e3dc-4f6d-8880-823011dc535a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:06.995024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecb17a-582d-455d-995a-863d15218eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:06.996409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecb17a-582d-455d-995a-863d15218eaf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:06.996726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecb17a-582d-455d-995a-863d15218eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:06.997039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecb17a-582d-455d-995a-863d15218eaf None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:06.998742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecb17a-582d-455d-995a-863d15218eaf None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:07.000676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecb17a-582d-455d-995a-863d15218eaf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:07.003027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 61/129] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:06 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:07.004301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecb17a-582d-455d-995a-863d15218eaf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:07.028130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27d9a2-f933-4eeb-8593-74a4bdd95d17 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:07.066152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c52cb-42f0-41c2-a144-e07b8234646a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:07.066939 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c52cb-42f0-41c2-a144-e07b8234646a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:07.067231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c52cb-42f0-41c2-a144-e07b8234646a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:07.067516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c52cb-42f0-41c2-a144-e07b8234646a None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:07.068852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c52cb-42f0-41c2-a144-e07b8234646a None admin] RBAC: Adding query filter params (name=member) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:07.069172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c52cb-42f0-41c2-a144-e07b8234646a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:07.070564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c52cb-42f0-41c2-a144-e07b8234646a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:07.079620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 69/130] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:07.080448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c52cb-42f0-41c2-a144-e07b8234646a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:07.141302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:07.142165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecb17a-582d-455d-995a-863d15218eaf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:07.162534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:07.180734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6244c9-bcb7-439f-84b1-53b565421dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:07.181726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6244c9-bcb7-439f-84b1-53b565421dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:07.182018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6244c9-bcb7-439f-84b1-53b565421dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:07.182298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6244c9-bcb7-439f-84b1-53b565421dd3 None admin] PATH_INFO: `/v3/users/e64e34286da842d2abb6f973aee3a086` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:07.183517 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6244c9-bcb7-439f-84b1-53b565421dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=e64e34286da842d2abb6f973aee3a086)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:07.188123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6244c9-bcb7-439f-84b1-53b565421dd3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:07.188123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 62/131] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:07 2022] GET /identity/v3/users/e64e34286da842d2abb6f973aee3a086 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:07.188123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6244c9-bcb7-439f-84b1-53b565421dd3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:07.242788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c52cb-42f0-41c2-a144-e07b8234646a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:07.315247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d5529-b8cd-49b6-9275-ad65c6008a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:07.317196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d5529-b8cd-49b6-9275-ad65c6008a19 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:07.318480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d5529-b8cd-49b6-9275-ad65c6008a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:07.318967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d5529-b8cd-49b6-9275-ad65c6008a19 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/users/45ceaef67f814339b45f2d297054bedf/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:07.322152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d5529-b8cd-49b6-9275-ad65c6008a19 None admin] RBAC: Authorizing `identity:create_grant(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user_id=45ceaef67f814339b45f2d297054bedf, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:07.323234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6244c9-bcb7-439f-84b1-53b565421dd3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:07.324103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d5529-b8cd-49b6-9275-ad65c6008a19 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:07.360386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e5937-4dbc-40e6-9e4a-8800e66d7cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:07.361366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e5937-4dbc-40e6-9e4a-8800e66d7cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:07.361665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e5937-4dbc-40e6-9e4a-8800e66d7cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:07.361996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e5937-4dbc-40e6-9e4a-8800e66d7cf1 None admin] PATH_INFO: `/v3/users/e64e34286da842d2abb6f973aee3a086` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:07.363247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e5937-4dbc-40e6-9e4a-8800e66d7cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=e64e34286da842d2abb6f973aee3a086)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:07.365290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e5937-4dbc-40e6-9e4a-8800e66d7cf1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:07.367337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 63/132] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:07 2022] GET /identity/v3/users/e64e34286da842d2abb6f973aee3a086 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:07.371948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e5937-4dbc-40e6-9e4a-8800e66d7cf1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:07.433781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:07.441835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 70/133] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:50:07 2022] PUT /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/users/45ceaef67f814339b45f2d297054bedf/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:07.445162 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66462764-d3a9-4967-aa11-03b3e91b5e75 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:07.451618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66462764-d3a9-4967-aa11-03b3e91b5e75 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:07.451618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66462764-d3a9-4967-aa11-03b3e91b5e75 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:07.451618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 71/134] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:07.451618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66462764-d3a9-4967-aa11-03b3e91b5e75 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:07.495135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e5937-4dbc-40e6-9e4a-8800e66d7cf1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:07.543061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d15c0f-1f87-49ac-bc71-01dcd68ab403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:07.544216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d15c0f-1f87-49ac-bc71-01dcd68ab403 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:07.544522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d15c0f-1f87-49ac-bc71-01dcd68ab403 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:07.544852 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d15c0f-1f87-49ac-bc71-01dcd68ab403 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:07.546095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d15c0f-1f87-49ac-bc71-01dcd68ab403 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:07.548760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d15c0f-1f87-49ac-bc71-01dcd68ab403 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:07.550680 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 64/135] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:07 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:07.556804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f7d5a-62a8-466c-989d-95af428a00e1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:07.556804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f7d5a-62a8-466c-989d-95af428a00e1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:07.556804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f7d5a-62a8-466c-989d-95af428a00e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:07.578146 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-318f7d5a-62a8-466c-989d-95af428a00e1 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:07.580205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-318f7d5a-62a8-466c-989d-95af428a00e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:07.590312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66462764-d3a9-4967-aa11-03b3e91b5e75 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:07.594094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:07.632112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561db42-4ff4-4984-9ec1-bb1376035fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:07.633427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561db42-4ff4-4984-9ec1-bb1376035fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:07.633933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561db42-4ff4-4984-9ec1-bb1376035fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:07.634420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561db42-4ff4-4984-9ec1-bb1376035fa2 None admin] PATH_INFO: `/v3/projects/e715b2d9555946578a479c254f7ef8dd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:07.636224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561db42-4ff4-4984-9ec1-bb1376035fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=e715b2d9555946578a479c254f7ef8dd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:07.638013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561db42-4ff4-4984-9ec1-bb1376035fa2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:07.640367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 72/136] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:07 2022] GET /identity/v3/projects/e715b2d9555946578a479c254f7ef8dd => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:07.641343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561db42-4ff4-4984-9ec1-bb1376035fa2 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:07.672006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 65/137] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:07.673355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f7d5a-62a8-466c-989d-95af428a00e1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:07.748698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:07.762349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561db42-4ff4-4984-9ec1-bb1376035fa2 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:07.803600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9e428-f958-462a-9c5d-e2d30c3e1d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:07.804586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9e428-f958-462a-9c5d-e2d30c3e1d50 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:07.804977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9e428-f958-462a-9c5d-e2d30c3e1d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:07.805501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9e428-f958-462a-9c5d-e2d30c3e1d50 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/users/e64e34286da842d2abb6f973aee3a086/roles/2991af796dcd45e2b86208339339b977` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:07.807719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9e428-f958-462a-9c5d-e2d30c3e1d50 None admin] RBAC: Authorizing `identity:create_grant(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user_id=e64e34286da842d2abb6f973aee3a086, role_id=2991af796dcd45e2b86208339339b977)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:07.809311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9e428-f958-462a-9c5d-e2d30c3e1d50 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:07.811854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f7d5a-62a8-466c-989d-95af428a00e1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:07.847610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5962b24-fa19-4d46-a34b-d87bcbbae97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:07.848811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5962b24-fa19-4d46-a34b-d87bcbbae97c None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:07.849848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5962b24-fa19-4d46-a34b-d87bcbbae97c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:07.850318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5962b24-fa19-4d46-a34b-d87bcbbae97c None admin] PATH_INFO: `/v3/projects/e715b2d9555946578a479c254f7ef8dd/users/e64e34286da842d2abb6f973aee3a086/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:07.852954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5962b24-fa19-4d46-a34b-d87bcbbae97c None admin] RBAC: Authorizing `identity:create_grant(project_id=e715b2d9555946578a479c254f7ef8dd, user_id=e64e34286da842d2abb6f973aee3a086, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:07.854654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5962b24-fa19-4d46-a34b-d87bcbbae97c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:07.887232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 73/138] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:50:07 2022] PUT /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/users/e64e34286da842d2abb6f973aee3a086/roles/2991af796dcd45e2b86208339339b977 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:07.888566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9e428-f958-462a-9c5d-e2d30c3e1d50 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:07.925670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:07.947091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 66/139] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:50:07 2022] PUT /identity/v3/projects/e715b2d9555946578a479c254f7ef8dd/users/e64e34286da842d2abb6f973aee3a086/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:08.018568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9e428-f958-462a-9c5d-e2d30c3e1d50 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:08.058117 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f941c-da70-4313-b44b-03aa007cbf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:08.058935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f941c-da70-4313-b44b-03aa007cbf5a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:08.059229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f941c-da70-4313-b44b-03aa007cbf5a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:08.059501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f941c-da70-4313-b44b-03aa007cbf5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:08.060798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f941c-da70-4313-b44b-03aa007cbf5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:08.062981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f941c-da70-4313-b44b-03aa007cbf5a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:08.064697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 74/140] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:08.070583 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5962b24-fa19-4d46-a34b-d87bcbbae97c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:08.187904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5962b24-fa19-4d46-a34b-d87bcbbae97c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:08.202979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:08.227153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60197f9e-9bb1-4f27-b130-e8ac04139467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:08.227891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60197f9e-9bb1-4f27-b130-e8ac04139467 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:08.228184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60197f9e-9bb1-4f27-b130-e8ac04139467 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:08.228451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60197f9e-9bb1-4f27-b130-e8ac04139467 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:08.229222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60197f9e-9bb1-4f27-b130-e8ac04139467 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:08.230770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60197f9e-9bb1-4f27-b130-e8ac04139467 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:08.330362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 67/141] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:08 2022] POST /identity/v3/users => generated 283 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:08.573033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e14ca-066c-49ea-844a-e36d0ab34307 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:08.573700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e14ca-066c-49ea-844a-e36d0ab34307 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:08.573700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e14ca-066c-49ea-844a-e36d0ab34307 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:08.575215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 75/142] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:08.580238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24174e-de11-4fba-aaa1-e91279099d81 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:08.580537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24174e-de11-4fba-aaa1-e91279099d81 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:08.580856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24174e-de11-4fba-aaa1-e91279099d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:08.598860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7c24174e-de11-4fba-aaa1-e91279099d81 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:08.600087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7c24174e-de11-4fba-aaa1-e91279099d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:08.696123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 68/143] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:08.730021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e14ca-066c-49ea-844a-e36d0ab34307 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:08.852108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:08.853092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e14ca-066c-49ea-844a-e36d0ab34307 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:08.895953 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ff359-81d5-40d2-86e9-373e94d68e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:08.896885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ff359-81d5-40d2-86e9-373e94d68e5e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:08.897174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ff359-81d5-40d2-86e9-373e94d68e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:08.897446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ff359-81d5-40d2-86e9-373e94d68e5e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:08.911217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ff359-81d5-40d2-86e9-373e94d68e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-591ff359-81d5-40d2-86e9-373e94d68e5e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:08.919152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:08.920926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:08.921669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 76/144] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:08.926267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24174e-de11-4fba-aaa1-e91279099d81 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:09.042272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:09.046194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24174e-de11-4fba-aaa1-e91279099d81 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:09.086349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da969917-27b7-405c-a86c-23e54ff5cf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:09.087035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da969917-27b7-405c-a86c-23e54ff5cf73 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:09.087321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da969917-27b7-405c-a86c-23e54ff5cf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:09.087648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da969917-27b7-405c-a86c-23e54ff5cf73 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:09.088441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da969917-27b7-405c-a86c-23e54ff5cf73 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:09.089209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da969917-27b7-405c-a86c-23e54ff5cf73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:09.091445 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da969917-27b7-405c-a86c-23e54ff5cf73 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:09.106348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 69/145] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:09.107859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ff359-81d5-40d2-86e9-373e94d68e5e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:09.230887 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ff359-81d5-40d2-86e9-373e94d68e5e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:09.281926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401bda4-7cee-47fc-9083-fe369a547c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:09.283139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401bda4-7cee-47fc-9083-fe369a547c12 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:09.283620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401bda4-7cee-47fc-9083-fe369a547c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:09.284112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401bda4-7cee-47fc-9083-fe369a547c12 None admin] PATH_INFO: `/v3/users/8c64f203641c4524840aa140fe420816` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:09.292884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401bda4-7cee-47fc-9083-fe369a547c12 None admin] RBAC: Authorizing `identity:get_user(user_id=8c64f203641c4524840aa140fe420816)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:09.294555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401bda4-7cee-47fc-9083-fe369a547c12 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:09.296426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 77/146] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:09 2022] GET /identity/v3/users/8c64f203641c4524840aa140fe420816 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:09.302045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da969917-27b7-405c-a86c-23e54ff5cf73 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:09.430495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da969917-27b7-405c-a86c-23e54ff5cf73 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:09.469047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47731de-f698-4082-b33e-29d999c555b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:09.469972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47731de-f698-4082-b33e-29d999c555b6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:09.470257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47731de-f698-4082-b33e-29d999c555b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:09.470652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47731de-f698-4082-b33e-29d999c555b6 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:09.471876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47731de-f698-4082-b33e-29d999c555b6 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:09.473378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47731de-f698-4082-b33e-29d999c555b6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:09.475736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 70/147] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:09 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:09.481752 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401bda4-7cee-47fc-9083-fe369a547c12 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:09.537507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:09.653050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401bda4-7cee-47fc-9083-fe369a547c12 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:09.705933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0072f2-b89c-4824-83b9-0bf06b5f2639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:09.707190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0072f2-b89c-4824-83b9-0bf06b5f2639 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:09.707684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0072f2-b89c-4824-83b9-0bf06b5f2639 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:09.708168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0072f2-b89c-4824-83b9-0bf06b5f2639 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/users/8c64f203641c4524840aa140fe420816/roles/583fe587d8d94b0fad411aeefb4181e0` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:09.714354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0072f2-b89c-4824-83b9-0bf06b5f2639 None admin] RBAC: Authorizing `identity:create_grant(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user_id=8c64f203641c4524840aa140fe420816, role_id=583fe587d8d94b0fad411aeefb4181e0)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:09.716216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0072f2-b89c-4824-83b9-0bf06b5f2639 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:09.728387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96bfba-4a08-4ac0-bb36-b32f90f4666f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:09.728699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96bfba-4a08-4ac0-bb36-b32f90f4666f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:09.728974 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96bfba-4a08-4ac0-bb36-b32f90f4666f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:09.730559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 71/148] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:50:09.736381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976a27c-2ad1-410c-8a08-4912e3183879 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:09.736715 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976a27c-2ad1-410c-8a08-4912e3183879 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:09.737055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976a27c-2ad1-410c-8a08-4912e3183879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:09.757947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1976a27c-2ad1-410c-8a08-4912e3183879 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:09.759201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1976a27c-2ad1-410c-8a08-4912e3183879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:09.790050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:09.797631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 78/149] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:50:09 2022] PUT /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/users/8c64f203641c4524840aa140fe420816/roles/583fe587d8d94b0fad411aeefb4181e0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:09.858988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 72/150] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:09.898953 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0072f2-b89c-4824-83b9-0bf06b5f2639 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:09.994185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:10.008838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0072f2-b89c-4824-83b9-0bf06b5f2639 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:10.048072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a8cae-8921-4e73-9c64-957f178cd8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:10.049009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a8cae-8921-4e73-9c64-957f178cd8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:10.049301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a8cae-8921-4e73-9c64-957f178cd8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:10.049577 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a8cae-8921-4e73-9c64-957f178cd8e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:10.064465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a8cae-8921-4e73-9c64-957f178cd8e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-522a8cae-8921-4e73-9c64-957f178cd8e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:10.073088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:10.073977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:10.074725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 79/151] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:10.083266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976a27c-2ad1-410c-8a08-4912e3183879 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:10.198349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976a27c-2ad1-410c-8a08-4912e3183879 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:10.240862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d41e8d-7dcc-4af6-bbfb-191bc96d3080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:10.241550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d41e8d-7dcc-4af6-bbfb-191bc96d3080 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:10.241834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d41e8d-7dcc-4af6-bbfb-191bc96d3080 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:10.242112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d41e8d-7dcc-4af6-bbfb-191bc96d3080 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:10.242875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d41e8d-7dcc-4af6-bbfb-191bc96d3080 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:10.243180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d41e8d-7dcc-4af6-bbfb-191bc96d3080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:10.244625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d41e8d-7dcc-4af6-bbfb-191bc96d3080 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:10.257186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 73/152] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:10.265879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a8cae-8921-4e73-9c64-957f178cd8e6 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:10.460404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a8cae-8921-4e73-9c64-957f178cd8e6 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:10.509634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b555b90-cde1-4107-b56b-4ba71bf1de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:10.510497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b555b90-cde1-4107-b56b-4ba71bf1de40 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:10.510785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b555b90-cde1-4107-b56b-4ba71bf1de40 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:10.511121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b555b90-cde1-4107-b56b-4ba71bf1de40 None admin] PATH_INFO: `/v3/users/e64e34286da842d2abb6f973aee3a086` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:10.512423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b555b90-cde1-4107-b56b-4ba71bf1de40 None admin] RBAC: Authorizing `identity:get_user(user_id=e64e34286da842d2abb6f973aee3a086)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:10.513955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b555b90-cde1-4107-b56b-4ba71bf1de40 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:10.515948 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 80/153] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:10 2022] GET /identity/v3/users/e64e34286da842d2abb6f973aee3a086 => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:10.521297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d41e8d-7dcc-4af6-bbfb-191bc96d3080 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:10.567559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:10.671263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d41e8d-7dcc-4af6-bbfb-191bc96d3080 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:10.716219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d7a6d-2e13-476b-a49e-3989554e8d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:10.716893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d7a6d-2e13-476b-a49e-3989554e8d2f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:10.717216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d7a6d-2e13-476b-a49e-3989554e8d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:10.717440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d7a6d-2e13-476b-a49e-3989554e8d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:10.794803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d7a6d-2e13-476b-a49e-3989554e8d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:10.796203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d7a6d-2e13-476b-a49e-3989554e8d2f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:10.822672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:10.921555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 74/154] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:10.930263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b555b90-cde1-4107-b56b-4ba71bf1de40 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:10.963255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:11.091254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b555b90-cde1-4107-b56b-4ba71bf1de40 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:11.142035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa61dc-6bfd-4ec5-946d-639192995a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:11.143265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa61dc-6bfd-4ec5-946d-639192995a66 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:11.143754 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa61dc-6bfd-4ec5-946d-639192995a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:11.144224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa61dc-6bfd-4ec5-946d-639192995a66 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:11.149507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa61dc-6bfd-4ec5-946d-639192995a66 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:11.151274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa61dc-6bfd-4ec5-946d-639192995a66 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:11.153277 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 81/155] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:10 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:11.159333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d7a6d-2e13-476b-a49e-3989554e8d2f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:11.275860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b31b0-e9ed-48c0-a250-fcd108e4f346 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:11.276500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b31b0-e9ed-48c0-a250-fcd108e4f346 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:11.277102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b31b0-e9ed-48c0-a250-fcd108e4f346 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:11.278662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 82/156] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:11.294403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c9ffa-1c9d-4913-9ea0-8d742b1f44bd None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:11.294937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c9ffa-1c9d-4913-9ea0-8d742b1f44bd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:11.295419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c9ffa-1c9d-4913-9ea0-8d742b1f44bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:11.357941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-571c9ffa-1c9d-4913-9ea0-8d742b1f44bd None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:11.359489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-571c9ffa-1c9d-4913-9ea0-8d742b1f44bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:11.397661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d7a6d-2e13-476b-a49e-3989554e8d2f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:11.436240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:11.481494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21444c-2cee-479f-a942-418e2293928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:11.482399 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21444c-2cee-479f-a942-418e2293928d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:11.482905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21444c-2cee-479f-a942-418e2293928d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:11.483314 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21444c-2cee-479f-a942-418e2293928d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:11.484302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21444c-2cee-479f-a942-418e2293928d None admin] RBAC: Adding query filter params (role.id=61e62f29eafd45c8b9fb99205a60bc4e, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user.id=e64e34286da842d2abb6f973aee3a086) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:11.484750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21444c-2cee-479f-a942-418e2293928d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:11.490259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21444c-2cee-479f-a942-418e2293928d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:11.501201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 75/157] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:11 2022] GET /identity/v3/role_assignments?user.id=e64e34286da842d2abb6f973aee3a086&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 608 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:11.502752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eca4f-2a48-4948-8523-bfce7ff5d566 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:11.503165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eca4f-2a48-4948-8523-bfce7ff5d566 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:11.503560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eca4f-2a48-4948-8523-bfce7ff5d566 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:11.505614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 76/158] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:11.518736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 83/159] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:11.526382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0415b52-03af-4a77-900a-694c3d3f0d0c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:11.527324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0415b52-03af-4a77-900a-694c3d3f0d0c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:11.527805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0415b52-03af-4a77-900a-694c3d3f0d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:11.569909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c9ffa-1c9d-4913-9ea0-8d742b1f44bd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:11.581923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c0415b52-03af-4a77-900a-694c3d3f0d0c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:11.584291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c0415b52-03af-4a77-900a-694c3d3f0d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:11.596145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:11.770554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c9ffa-1c9d-4913-9ea0-8d742b1f44bd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:11.774152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 77/160] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:11.835627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d449aba-a4d0-436c-ad8b-00934f9af466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:11.836586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d449aba-a4d0-436c-ad8b-00934f9af466 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:11.836901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d449aba-a4d0-436c-ad8b-00934f9af466 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:11.837245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d449aba-a4d0-436c-ad8b-00934f9af466 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:11.855529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d449aba-a4d0-436c-ad8b-00934f9af466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:11.861463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0415b52-03af-4a77-900a-694c3d3f0d0c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-4d449aba-a4d0-436c-ad8b-00934f9af466 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:11.867143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:11.868032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:11.868915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 84/161] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:11.874318 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d449aba-a4d0-436c-ad8b-00934f9af466 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:12.012569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:12.018096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d449aba-a4d0-436c-ad8b-00934f9af466 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:12.058814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0415b52-03af-4a77-900a-694c3d3f0d0c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:12.073523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fc97b-aa51-4440-b7e9-554483d3450e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:12.074144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fc97b-aa51-4440-b7e9-554483d3450e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:12.074432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fc97b-aa51-4440-b7e9-554483d3450e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:12.074712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fc97b-aa51-4440-b7e9-554483d3450e None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:12.075469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fc97b-aa51-4440-b7e9-554483d3450e None admin] RBAC: Adding query filter params (name=member) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:12.075952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fc97b-aa51-4440-b7e9-554483d3450e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:12.077544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fc97b-aa51-4440-b7e9-554483d3450e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:12.084096 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:12.087143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 85/162] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:12.088558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b5af3-8e60-439b-82c5-46e5fd5c2936 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:12.093282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b5af3-8e60-439b-82c5-46e5fd5c2936 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:12.093592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b5af3-8e60-439b-82c5-46e5fd5c2936 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:12.095151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 86/163] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:12.096485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b5af3-8e60-439b-82c5-46e5fd5c2936 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:12.109992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022e370-b881-471e-8908-fb732474a770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:12.111133 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022e370-b881-471e-8908-fb732474a770 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:12.111610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022e370-b881-471e-8908-fb732474a770 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:12.112094 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022e370-b881-471e-8908-fb732474a770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:12.135235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022e370-b881-471e-8908-fb732474a770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c022e370-b881-471e-8908-fb732474a770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:12.144237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:12.149343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:12.150097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:12.158409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 78/164] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:50:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:12.160428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05b9ed-2103-4828-b6f1-54f5842c0368 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:12.161986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05b9ed-2103-4828-b6f1-54f5842c0368 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:12.162943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05b9ed-2103-4828-b6f1-54f5842c0368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:12.199236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6d05b9ed-2103-4828-b6f1-54f5842c0368 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:12.201563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6d05b9ed-2103-4828-b6f1-54f5842c0368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:12.335729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b5af3-8e60-439b-82c5-46e5fd5c2936 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:12.356488 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 79/165] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:12.357914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05b9ed-2103-4828-b6f1-54f5842c0368 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:12.407878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0e6f2-be41-4a93-8e7e-de3aa70552d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:12.409053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0e6f2-be41-4a93-8e7e-de3aa70552d8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:12.409335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0e6f2-be41-4a93-8e7e-de3aa70552d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:12.409606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0e6f2-be41-4a93-8e7e-de3aa70552d8 None admin] PATH_INFO: `/v3/users/e64e34286da842d2abb6f973aee3a086` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:12.411987 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0e6f2-be41-4a93-8e7e-de3aa70552d8 None admin] RBAC: Authorizing `identity:get_user(user_id=e64e34286da842d2abb6f973aee3a086)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:12.413764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0e6f2-be41-4a93-8e7e-de3aa70552d8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:12.416185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 87/166] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:12 2022] GET /identity/v3/users/e64e34286da842d2abb6f973aee3a086 => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:12.416810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0e6f2-be41-4a93-8e7e-de3aa70552d8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:12.427142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:12.517073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05b9ed-2103-4828-b6f1-54f5842c0368 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:12.523934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:12.592735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8031baf-a114-41da-bb87-427edce4ba47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:12.594207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8031baf-a114-41da-bb87-427edce4ba47 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:12.594711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8031baf-a114-41da-bb87-427edce4ba47 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:12.595185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8031baf-a114-41da-bb87-427edce4ba47 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:12.596135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8031baf-a114-41da-bb87-427edce4ba47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:12.596637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8031baf-a114-41da-bb87-427edce4ba47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:12.598563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8031baf-a114-41da-bb87-427edce4ba47 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:12.612599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 80/167] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:12.613689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8031baf-a114-41da-bb87-427edce4ba47 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:12.651808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0e6f2-be41-4a93-8e7e-de3aa70552d8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:12.719872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219c65b-2921-4ca0-9faa-79772ac17b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:12.721139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219c65b-2921-4ca0-9faa-79772ac17b1b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:12.721695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219c65b-2921-4ca0-9faa-79772ac17b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:12.722167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219c65b-2921-4ca0-9faa-79772ac17b1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:12.736262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219c65b-2921-4ca0-9faa-79772ac17b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-a219c65b-2921-4ca0-9faa-79772ac17b1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:12.745341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:12.746242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:12.746994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:12.749756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 88/168] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:12.750751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219c65b-2921-4ca0-9faa-79772ac17b1b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:12.810300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8031baf-a114-41da-bb87-427edce4ba47 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:12.822282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:12.871932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf4d75-1443-41df-af36-f221bb76b884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:12.872512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf4d75-1443-41df-af36-f221bb76b884 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:12.872861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf4d75-1443-41df-af36-f221bb76b884 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:12.873192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf4d75-1443-41df-af36-f221bb76b884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:12.910447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:12.918982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219c65b-2921-4ca0-9faa-79772ac17b1b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:12.947504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf4d75-1443-41df-af36-f221bb76b884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:12.949360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf4d75-1443-41df-af36-f221bb76b884 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:12.985521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb67a7-4560-4908-9e3e-a5e952ab23c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:12.986414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb67a7-4560-4908-9e3e-a5e952ab23c9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:12.986696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb67a7-4560-4908-9e3e-a5e952ab23c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:12.986968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb67a7-4560-4908-9e3e-a5e952ab23c9 None admin] PATH_INFO: `/v3/users/45ceaef67f814339b45f2d297054bedf` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:12.988198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb67a7-4560-4908-9e3e-a5e952ab23c9 None admin] RBAC: Authorizing `identity:get_user(user_id=45ceaef67f814339b45f2d297054bedf)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:12.990245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb67a7-4560-4908-9e3e-a5e952ab23c9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:12.991951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 89/169] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:12 2022] GET /identity/v3/users/45ceaef67f814339b45f2d297054bedf => generated 247 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:12.992721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb67a7-4560-4908-9e3e-a5e952ab23c9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:13.057818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 81/170] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:13.058506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf4d75-1443-41df-af36-f221bb76b884 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:13.104124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb67a7-4560-4908-9e3e-a5e952ab23c9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:13.138188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:13.138379 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:13.149133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcb5a6-5aaa-4913-b815-958c746bf769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:13.149850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcb5a6-5aaa-4913-b815-958c746bf769 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.153559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcb5a6-5aaa-4913-b815-958c746bf769 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.153559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcb5a6-5aaa-4913-b815-958c746bf769 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.153559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcb5a6-5aaa-4913-b815-958c746bf769 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:13.153559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcb5a6-5aaa-4913-b815-958c746bf769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:13.153559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcb5a6-5aaa-4913-b815-958c746bf769 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:13.175049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf4d75-1443-41df-af36-f221bb76b884 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:13.179329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 90/171] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:13.180172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcb5a6-5aaa-4913-b815-958c746bf769 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:13.215264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c3ef8-f722-4756-a10a-8750ed30b269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:13.215967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3ef8-f722-4756-a10a-8750ed30b269 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.216258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3ef8-f722-4756-a10a-8750ed30b269 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.216533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3ef8-f722-4756-a10a-8750ed30b269 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.323548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c3ef8-f722-4756-a10a-8750ed30b269 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:13.325084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c3ef8-f722-4756-a10a-8750ed30b269 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:13.382851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcb5a6-5aaa-4913-b815-958c746bf769 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:13.451501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d13a7-784f-47d3-ae2d-e85b10afb021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:13.452377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d13a7-784f-47d3-ae2d-e85b10afb021 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.452664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d13a7-784f-47d3-ae2d-e85b10afb021 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.453028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d13a7-784f-47d3-ae2d-e85b10afb021 None admin] PATH_INFO: `/v3/projects/e715b2d9555946578a479c254f7ef8dd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.457268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d13a7-784f-47d3-ae2d-e85b10afb021 None admin] RBAC: Authorizing `identity:get_project(project_id=e715b2d9555946578a479c254f7ef8dd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:13.459442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d13a7-784f-47d3-ae2d-e85b10afb021 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:13.461562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 91/172] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:13 2022] GET /identity/v3/projects/e715b2d9555946578a479c254f7ef8dd => generated 312 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:13.463006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d13a7-784f-47d3-ae2d-e85b10afb021 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:13.465939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:13.538000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 82/173] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:13.538748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c3ef8-f722-4756-a10a-8750ed30b269 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:13.570725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:13.624918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d13a7-784f-47d3-ae2d-e85b10afb021 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:13.649947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c3ef8-f722-4756-a10a-8750ed30b269 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:13.666944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fc4f7-8309-4c67-82a1-38a5ba7102ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:13.667817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc4f7-8309-4c67-82a1-38a5ba7102ec None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.668100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc4f7-8309-4c67-82a1-38a5ba7102ec None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.668372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc4f7-8309-4c67-82a1-38a5ba7102ec None admin] PATH_INFO: `/v3/users/e64e34286da842d2abb6f973aee3a086` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.669578 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fc4f7-8309-4c67-82a1-38a5ba7102ec None admin] RBAC: Authorizing `identity:get_user(user_id=e64e34286da842d2abb6f973aee3a086)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:13.671009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fc4f7-8309-4c67-82a1-38a5ba7102ec None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:13.672626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 92/174] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:13 2022] GET /identity/v3/users/e64e34286da842d2abb6f973aee3a086 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:13.673831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b39f8-ef68-4bf1-aff5-68a55ccccf40 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.674111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b39f8-ef68-4bf1-aff5-68a55ccccf40 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.674384 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b39f8-ef68-4bf1-aff5-68a55ccccf40 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.675590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 93/175] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:13.676737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45bb7-ba09-43c6-b8ee-ca60e25ce653 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.677731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45bb7-ba09-43c6-b8ee-ca60e25ce653 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.677731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45bb7-ba09-43c6-b8ee-ca60e25ce653 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.678555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 94/176] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:13.682674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf45bb7-ba09-43c6-b8ee-ca60e25ce653 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:13.694610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b2491-a546-451d-84df-1bed84140285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:13.695289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b2491-a546-451d-84df-1bed84140285 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.695860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b2491-a546-451d-84df-1bed84140285 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.696266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b2491-a546-451d-84df-1bed84140285 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.697061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b2491-a546-451d-84df-1bed84140285 None admin] RBAC: Adding query filter params (role.id=61e62f29eafd45c8b9fb99205a60bc4e, scope.project.id=e715b2d9555946578a479c254f7ef8dd, user.id=e64e34286da842d2abb6f973aee3a086) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:13.697373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b2491-a546-451d-84df-1bed84140285 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:13.699087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b2491-a546-451d-84df-1bed84140285 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:13.712270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 83/177] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:13 2022] GET /identity/v3/role_assignments?user.id=e64e34286da842d2abb6f973aee3a086&scope.project.id=e715b2d9555946578a479c254f7ef8dd&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:13.712270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853cabf-72a8-4709-bbc5-75444d160779 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.712270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853cabf-72a8-4709-bbc5-75444d160779 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.712270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853cabf-72a8-4709-bbc5-75444d160779 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.712270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 84/178] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:13.712270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853cabf-72a8-4709-bbc5-75444d160779 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:13.748519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:13.826140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf45bb7-ba09-43c6-b8ee-ca60e25ce653 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:13.838776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853cabf-72a8-4709-bbc5-75444d160779 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:13.862638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:13.869216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704455ec-2baf-4be0-8a3d-bad553d956cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:13.870166 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704455ec-2baf-4be0-8a3d-bad553d956cb None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.870446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704455ec-2baf-4be0-8a3d-bad553d956cb None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.870718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704455ec-2baf-4be0-8a3d-bad553d956cb None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.872051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704455ec-2baf-4be0-8a3d-bad553d956cb None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:13.873524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704455ec-2baf-4be0-8a3d-bad553d956cb None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:13.875232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 95/179] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:13 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:13.876385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bd3f6-8130-478c-a667-0be39db8d43f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.876665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bd3f6-8130-478c-a667-0be39db8d43f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.878909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bd3f6-8130-478c-a667-0be39db8d43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.882688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f12301-4532-4785-9ddc-68b24c53d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:13.883885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f12301-4532-4785-9ddc-68b24c53d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.883885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f12301-4532-4785-9ddc-68b24c53d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.883885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f12301-4532-4785-9ddc-68b24c53d5d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.898042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-fb0bd3f6-8130-478c-a667-0be39db8d43f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:13.899251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fb0bd3f6-8130-478c-a667-0be39db8d43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:13.968110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f12301-4532-4785-9ddc-68b24c53d5d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:13.969544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f12301-4532-4785-9ddc-68b24c53d5d1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:13.977779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 96/180] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:13.979056 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648be30-e998-4026-bd12-506c344b5cb8 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:13.979316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648be30-e998-4026-bd12-506c344b5cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:13.979590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648be30-e998-4026-bd12-506c344b5cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:13.998290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-5648be30-e998-4026-bd12-506c344b5cb8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:13.999460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5648be30-e998-4026-bd12-506c344b5cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:14.006858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:14.066210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 97/181] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:14.080973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cca3a-dd55-44b3-bd61-e6babbf0e634 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.080973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cca3a-dd55-44b3-bd61-e6babbf0e634 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.080973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cca3a-dd55-44b3-bd61-e6babbf0e634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.082717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 85/182] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:14.084855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda2dd8-8374-4a12-a65e-0ddb95cc89ef None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.085313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda2dd8-8374-4a12-a65e-0ddb95cc89ef None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.085735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda2dd8-8374-4a12-a65e-0ddb95cc89ef None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.087786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 86/183] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:14.091989 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda2dd8-8374-4a12-a65e-0ddb95cc89ef None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:14.092066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-934cca3a-dd55-44b3-bd61-e6babbf0e634 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:14.092066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-934cca3a-dd55-44b3-bd61-e6babbf0e634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:14.147151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:14.160348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 98/184] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:14.163290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cca3a-dd55-44b3-bd61-e6babbf0e634 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:14.193767 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda2dd8-8374-4a12-a65e-0ddb95cc89ef None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:14.222079 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:14.227749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cea24-482b-4409-8663-782eb94f7c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:14.228429 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cea24-482b-4409-8663-782eb94f7c14 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.231548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cea24-482b-4409-8663-782eb94f7c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.231548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cea24-482b-4409-8663-782eb94f7c14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.231548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cea24-482b-4409-8663-782eb94f7c14 None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user.id=45ceaef67f814339b45f2d297054bedf) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:14.231548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cea24-482b-4409-8663-782eb94f7c14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:14.231841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cea24-482b-4409-8663-782eb94f7c14 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:14.237612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 87/185] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/role_assignments?user.id=45ceaef67f814339b45f2d297054bedf&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:14.238298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cea24-482b-4409-8663-782eb94f7c14 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:14.277060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cca3a-dd55-44b3-bd61-e6babbf0e634 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:14.320225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6585c-1437-4e9b-aaf4-1ed76a6e4583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:14.321065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6585c-1437-4e9b-aaf4-1ed76a6e4583 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.321349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6585c-1437-4e9b-aaf4-1ed76a6e4583 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.321619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6585c-1437-4e9b-aaf4-1ed76a6e4583 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.334239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6585c-1437-4e9b-aaf4-1ed76a6e4583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-04d6585c-1437-4e9b-aaf4-1ed76a6e4583 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:14.342127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:14.343269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 99/186] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:14.345552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6585c-1437-4e9b-aaf4-1ed76a6e4583 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:14.370671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cea24-482b-4409-8663-782eb94f7c14 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:14.401158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:14.413376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f72e9d-e05f-4d04-bf28-5b47dca7b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:14.414844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f72e9d-e05f-4d04-bf28-5b47dca7b5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.414844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f72e9d-e05f-4d04-bf28-5b47dca7b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.415046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f72e9d-e05f-4d04-bf28-5b47dca7b5d6 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.416151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f72e9d-e05f-4d04-bf28-5b47dca7b5d6 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:14.417659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f72e9d-e05f-4d04-bf28-5b47dca7b5d6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:14.419408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 88/187] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:14.420751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ef59e-a242-4a39-8a46-dbdda5f83827 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.421073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ef59e-a242-4a39-8a46-dbdda5f83827 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.421534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ef59e-a242-4a39-8a46-dbdda5f83827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.441422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bb1ef59e-a242-4a39-8a46-dbdda5f83827 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:14.443365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bb1ef59e-a242-4a39-8a46-dbdda5f83827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:14.469827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6585c-1437-4e9b-aaf4-1ed76a6e4583 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:14.507066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6e369-15fa-4da6-9998-e08e7bd6c02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:14.507775 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6e369-15fa-4da6-9998-e08e7bd6c02c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.508053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6e369-15fa-4da6-9998-e08e7bd6c02c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.508330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6e369-15fa-4da6-9998-e08e7bd6c02c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.508593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:14.518437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6e369-15fa-4da6-9998-e08e7bd6c02c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:14.518572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 89/188] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:14.519300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ef59e-a242-4a39-8a46-dbdda5f83827 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-d6a6e369-15fa-4da6-9998-e08e7bd6c02c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:14.526013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:14.526882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 100/189] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:14.528967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6e369-15fa-4da6-9998-e08e7bd6c02c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:14.619820 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ef59e-a242-4a39-8a46-dbdda5f83827 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:14.625676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6e369-15fa-4da6-9998-e08e7bd6c02c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:14.634429 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:14.652927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b5cfd-90e6-400a-a7e6-133ab78d2e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:14.653731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b5cfd-90e6-400a-a7e6-133ab78d2e2e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.654011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b5cfd-90e6-400a-a7e6-133ab78d2e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.654285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b5cfd-90e6-400a-a7e6-133ab78d2e2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.660054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df89dec-cf74-413b-b40b-3d9b4939f5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:14.660707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df89dec-cf74-413b-b40b-3d9b4939f5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.661013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df89dec-cf74-413b-b40b-3d9b4939f5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.661825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df89dec-cf74-413b-b40b-3d9b4939f5c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.662567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df89dec-cf74-413b-b40b-3d9b4939f5c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:14.662873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df89dec-cf74-413b-b40b-3d9b4939f5c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:14.664251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df89dec-cf74-413b-b40b-3d9b4939f5c6 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:14.664822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b5cfd-90e6-400a-a7e6-133ab78d2e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:14.672812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 101/190] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-399b5cfd-90e6-400a-a7e6-133ab78d2e2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:14.672923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:14.673759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:14.674535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:14.675121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df89dec-cf74-413b-b40b-3d9b4939f5c6 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:14.675186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 90/191] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:14.678202 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b5cfd-90e6-400a-a7e6-133ab78d2e2e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:14.730840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:14.787829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df89dec-cf74-413b-b40b-3d9b4939f5c6 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:14.788018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b5cfd-90e6-400a-a7e6-133ab78d2e2e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:14.820676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596769f4-4e28-45ea-82c8-f5905568c596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:14.821350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596769f4-4e28-45ea-82c8-f5905568c596 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.821636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596769f4-4e28-45ea-82c8-f5905568c596 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.821750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f6e49-a815-4e5c-8d9a-8f6ca9b61d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:14.821925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596769f4-4e28-45ea-82c8-f5905568c596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.822334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f6e49-a815-4e5c-8d9a-8f6ca9b61d57 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.822621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f6e49-a815-4e5c-8d9a-8f6ca9b61d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.822764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596769f4-4e28-45ea-82c8-f5905568c596 None admin] RBAC: Adding query filter params (role.id=2991af796dcd45e2b86208339339b977, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user.id=e64e34286da842d2abb6f973aee3a086) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:14.822903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f6e49-a815-4e5c-8d9a-8f6ca9b61d57 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.823082 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596769f4-4e28-45ea-82c8-f5905568c596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:14.823652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f6e49-a815-4e5c-8d9a-8f6ca9b61d57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:14.823963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f6e49-a815-4e5c-8d9a-8f6ca9b61d57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:14.824555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596769f4-4e28-45ea-82c8-f5905568c596 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:14.825439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f6e49-a815-4e5c-8d9a-8f6ca9b61d57 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:14.830755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 102/192] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/role_assignments?user.id=e64e34286da842d2abb6f973aee3a086&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=2991af796dcd45e2b86208339339b977 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:14.831371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596769f4-4e28-45ea-82c8-f5905568c596 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:14.837533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 91/193] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:14.838725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f6e49-a815-4e5c-8d9a-8f6ca9b61d57 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:14.872673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:14.944594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596769f4-4e28-45ea-82c8-f5905568c596 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:14.966957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f6e49-a815-4e5c-8d9a-8f6ca9b61d57 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:14.984989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dffe9-7586-4785-b77a-0bf9b176c796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:14.985187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:14.985521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dffe9-7586-4785-b77a-0bf9b176c796 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:14.985808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dffe9-7586-4785-b77a-0bf9b176c796 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:14.986084 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dffe9-7586-4785-b77a-0bf9b176c796 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:14.987704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043dffe9-7586-4785-b77a-0bf9b176c796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:14.989285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043dffe9-7586-4785-b77a-0bf9b176c796 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:14.991446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 103/194] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:14.992140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dffe9-7586-4785-b77a-0bf9b176c796 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:15.009622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc9528-e8e7-4be1-893c-65ba51ecc911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:15.011073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc9528-e8e7-4be1-893c-65ba51ecc911 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:15.011351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc9528-e8e7-4be1-893c-65ba51ecc911 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:15.011625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc9528-e8e7-4be1-893c-65ba51ecc911 None admin] PATH_INFO: `/v3/users/8c64f203641c4524840aa140fe420816` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:15.012848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc9528-e8e7-4be1-893c-65ba51ecc911 None admin] RBAC: Authorizing `identity:get_user(user_id=8c64f203641c4524840aa140fe420816)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:15.014297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc9528-e8e7-4be1-893c-65ba51ecc911 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:15.017083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 92/195] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/users/8c64f203641c4524840aa140fe420816 => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:15.021125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc9528-e8e7-4be1-893c-65ba51ecc911 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:15.109925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dffe9-7586-4785-b77a-0bf9b176c796 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:15.111355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:15.133630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc9528-e8e7-4be1-893c-65ba51ecc911 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:15.151625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0aff74-c322-41f9-83e1-63f54b13a757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:15.152287 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aff74-c322-41f9-83e1-63f54b13a757 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:15.152567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aff74-c322-41f9-83e1-63f54b13a757 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:15.152868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aff74-c322-41f9-83e1-63f54b13a757 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:15.153610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0aff74-c322-41f9-83e1-63f54b13a757 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:15.153912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0aff74-c322-41f9-83e1-63f54b13a757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:15.155398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0aff74-c322-41f9-83e1-63f54b13a757 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:15.163149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 104/196] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:15.163904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0aff74-c322-41f9-83e1-63f54b13a757 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:15.168912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620dd62-97ce-4cdb-912a-b02bad9f51cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:15.169782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620dd62-97ce-4cdb-912a-b02bad9f51cb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:15.170067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620dd62-97ce-4cdb-912a-b02bad9f51cb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:15.170342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620dd62-97ce-4cdb-912a-b02bad9f51cb None admin] PATH_INFO: `/v3/users/17ab4b315fb545808d2644766f27d8fb` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:15.183567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620dd62-97ce-4cdb-912a-b02bad9f51cb None admin] RBAC: Authorizing `identity:get_user(user_id=17ab4b315fb545808d2644766f27d8fb)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:15.185049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620dd62-97ce-4cdb-912a-b02bad9f51cb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:15.186699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 93/197] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:15 2022] GET /identity/v3/users/17ab4b315fb545808d2644766f27d8fb => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:15.187451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620dd62-97ce-4cdb-912a-b02bad9f51cb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:15.228064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:15.264100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0aff74-c322-41f9-83e1-63f54b13a757 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:15.285443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620dd62-97ce-4cdb-912a-b02bad9f51cb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:15.297342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b43e52-bd4f-4b5f-bb97-5bdea3ed4b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:15.298021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b43e52-bd4f-4b5f-bb97-5bdea3ed4b0d None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:15.298308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b43e52-bd4f-4b5f-bb97-5bdea3ed4b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:15.298590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b43e52-bd4f-4b5f-bb97-5bdea3ed4b0d None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:15.299303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b43e52-bd4f-4b5f-bb97-5bdea3ed4b0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:15.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b43e52-bd4f-4b5f-bb97-5bdea3ed4b0d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:15.321198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1c3ab-8075-4d46-94bd-d4b6f2c62cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:15.327552 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c3ab-8075-4d46-94bd-d4b6f2c62cdc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:15.327552 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c3ab-8075-4d46-94bd-d4b6f2c62cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:15.327552 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c3ab-8075-4d46-94bd-d4b6f2c62cdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:15.387398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1c3ab-8075-4d46-94bd-d4b6f2c62cdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:15.388821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1c3ab-8075-4d46-94bd-d4b6f2c62cdc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:15.404922 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 105/198] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:15 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:15.405867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b43e52-bd4f-4b5f-bb97-5bdea3ed4b0d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:15.441198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:15.443592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:15.510240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 94/199] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:15.510901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1c3ab-8075-4d46-94bd-d4b6f2c62cdc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:15.535696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b43e52-bd4f-4b5f-bb97-5bdea3ed4b0d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:15.574517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407530c-59d7-4efe-9080-6b492c30bcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:15.575398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407530c-59d7-4efe-9080-6b492c30bcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:15.575683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407530c-59d7-4efe-9080-6b492c30bcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:15.575957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407530c-59d7-4efe-9080-6b492c30bcd4 None admin] PATH_INFO: `/v3/users/17ab4b315fb545808d2644766f27d8fb` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:15.577632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407530c-59d7-4efe-9080-6b492c30bcd4 None admin] RBAC: Authorizing `identity:get_user(user_id=17ab4b315fb545808d2644766f27d8fb)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:15.579221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407530c-59d7-4efe-9080-6b492c30bcd4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:15.580252 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 106/200] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:15 2022] GET /identity/v3/users/17ab4b315fb545808d2644766f27d8fb => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:15.581024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407530c-59d7-4efe-9080-6b492c30bcd4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:15.631219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1c3ab-8075-4d46-94bd-d4b6f2c62cdc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:15.670240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097241e-f270-409d-aa65-638a7c55694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:15.670894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097241e-f270-409d-aa65-638a7c55694d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:15.671177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097241e-f270-409d-aa65-638a7c55694d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:15.671452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097241e-f270-409d-aa65-638a7c55694d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:15.684701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:15.686792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:15.703195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407530c-59d7-4efe-9080-6b492c30bcd4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:15.754277 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99b894-a239-456a-86df-a000f70692cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:15.755706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99b894-a239-456a-86df-a000f70692cf None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:15.755985 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99b894-a239-456a-86df-a000f70692cf None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:15.756260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99b894-a239-456a-86df-a000f70692cf None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:15.758351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99b894-a239-456a-86df-a000f70692cf None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:15.759810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99b894-a239-456a-86df-a000f70692cf None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:15.761897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 107/201] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:15 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:15.762622 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99b894-a239-456a-86df-a000f70692cf None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:15.763406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097241e-f270-409d-aa65-638a7c55694d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:15.765205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097241e-f270-409d-aa65-638a7c55694d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:15.892976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99b894-a239-456a-86df-a000f70692cf None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:15.902963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 95/202] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:15.904418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097241e-f270-409d-aa65-638a7c55694d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:15.930996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96e71d-2fc8-470f-b5db-efeb7de255a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:15.931571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96e71d-2fc8-470f-b5db-efeb7de255a3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:15.931854 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96e71d-2fc8-470f-b5db-efeb7de255a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:15.932127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96e71d-2fc8-470f-b5db-efeb7de255a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:15.956303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:15.961715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:16.017560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96e71d-2fc8-470f-b5db-efeb7de255a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:16.019032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96e71d-2fc8-470f-b5db-efeb7de255a3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:16.023954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097241e-f270-409d-aa65-638a7c55694d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:16.070128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c2bfc-f09b-4420-8228-7dfd790716b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:16.071513 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c2bfc-f09b-4420-8228-7dfd790716b2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:16.072006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c2bfc-f09b-4420-8228-7dfd790716b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:16.072496 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c2bfc-f09b-4420-8228-7dfd790716b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:16.074096 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c2bfc-f09b-4420-8228-7dfd790716b2 None admin] RBAC: Adding query filter params (role.id=583fe587d8d94b0fad411aeefb4181e0, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, user.id=8c64f203641c4524840aa140fe420816) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:16.074613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c2bfc-f09b-4420-8228-7dfd790716b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:16.076758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c2bfc-f09b-4420-8228-7dfd790716b2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:16.085519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 96/203] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:15 2022] GET /identity/v3/role_assignments?user.id=8c64f203641c4524840aa140fe420816&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=583fe587d8d94b0fad411aeefb4181e0 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:16.086482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c2bfc-f09b-4420-8228-7dfd790716b2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:16.206872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c2bfc-f09b-4420-8228-7dfd790716b2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:16.207042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 108/204] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:16.211928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:16.213058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96e71d-2fc8-470f-b5db-efeb7de255a3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:16.249936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbaf5df-bd3d-475f-a324-1e45dd4cfe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:16.251109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaf5df-bd3d-475f-a324-1e45dd4cfe46 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:16.251803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaf5df-bd3d-475f-a324-1e45dd4cfe46 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:16.252270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaf5df-bd3d-475f-a324-1e45dd4cfe46 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:16.253963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbaf5df-bd3d-475f-a324-1e45dd4cfe46 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:16.256038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbaf5df-bd3d-475f-a324-1e45dd4cfe46 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:16.257987 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 97/205] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:16 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:16.263906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbaf5df-bd3d-475f-a324-1e45dd4cfe46 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:16.335736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:16.393585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96e71d-2fc8-470f-b5db-efeb7de255a3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:16.424290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbaf5df-bd3d-475f-a324-1e45dd4cfe46 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:16.434489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd83aa-0742-4d6f-a6c9-3fbee3055890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:16.435477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd83aa-0742-4d6f-a6c9-3fbee3055890 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:16.435739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd83aa-0742-4d6f-a6c9-3fbee3055890 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:16.435949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd83aa-0742-4d6f-a6c9-3fbee3055890 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:16.437731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd83aa-0742-4d6f-a6c9-3fbee3055890 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:16.439196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd83aa-0742-4d6f-a6c9-3fbee3055890 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:16.441235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 109/206] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:16 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:16.446567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd83aa-0742-4d6f-a6c9-3fbee3055890 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:16.467776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99aa4b4-add5-4fba-b511-e76eeea4c5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:16.468362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aa4b4-add5-4fba-b511-e76eeea4c5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:16.468642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aa4b4-add5-4fba-b511-e76eeea4c5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:16.468940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aa4b4-add5-4fba-b511-e76eeea4c5c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:16.469776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99aa4b4-add5-4fba-b511-e76eeea4c5c1 None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b, user.id=17ab4b315fb545808d2644766f27d8fb) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:16.470088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99aa4b4-add5-4fba-b511-e76eeea4c5c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:16.471545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99aa4b4-add5-4fba-b511-e76eeea4c5c1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:16.480721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 98/207] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:16 2022] GET /identity/v3/role_assignments?user.id=17ab4b315fb545808d2644766f27d8fb&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:16.573376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd83aa-0742-4d6f-a6c9-3fbee3055890 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:16.601161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:16.611546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429a5b8-c0a3-47d9-8a3a-bd6dcae12292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:16.612216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429a5b8-c0a3-47d9-8a3a-bd6dcae12292 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:16.612499 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429a5b8-c0a3-47d9-8a3a-bd6dcae12292 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:16.612801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429a5b8-c0a3-47d9-8a3a-bd6dcae12292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:16.613648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429a5b8-c0a3-47d9-8a3a-bd6dcae12292 None admin] RBAC: Adding query filter params (role.id=2991af796dcd45e2b86208339339b977, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b, user.id=17ab4b315fb545808d2644766f27d8fb) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:16.613959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429a5b8-c0a3-47d9-8a3a-bd6dcae12292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:16.615450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429a5b8-c0a3-47d9-8a3a-bd6dcae12292 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:16.621846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 110/208] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:16 2022] GET /identity/v3/role_assignments?user.id=17ab4b315fb545808d2644766f27d8fb&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=2991af796dcd45e2b86208339339b977 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:17.637501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d1077-bd91-4c63-b820-c69fd68f94b2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:17.638005 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d1077-bd91-4c63-b820-c69fd68f94b2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:17.638005 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d1077-bd91-4c63-b820-c69fd68f94b2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:17.639595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 99/209] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:17.644917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e67097-b92c-4b83-82d4-58ca27c49769 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:17.646303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e67097-b92c-4b83-82d4-58ca27c49769 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:17.646805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e67097-b92c-4b83-82d4-58ca27c49769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:17.663600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-59e67097-b92c-4b83-82d4-58ca27c49769 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:17.665099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-59e67097-b92c-4b83-82d4-58ca27c49769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:17.674458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ecd54-2ba9-45fa-bb74-223e2e6f9c1e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:17.675391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ecd54-2ba9-45fa-bb74-223e2e6f9c1e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:17.675894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ecd54-2ba9-45fa-bb74-223e2e6f9c1e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:17.678185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 100/210] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:17.683475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5331e4-9286-4f8b-9dce-1586519340a7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:17.683975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5331e4-9286-4f8b-9dce-1586519340a7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:17.684485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5331e4-9286-4f8b-9dce-1586519340a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:17.701843 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cc5331e4-9286-4f8b-9dce-1586519340a7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:17.703288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cc5331e4-9286-4f8b-9dce-1586519340a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:17.736294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:17.747941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 111/211] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:17.772444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e67097-b92c-4b83-82d4-58ca27c49769 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:17.778925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 101/212] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:17.804334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5331e4-9286-4f8b-9dce-1586519340a7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:17.873089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e67097-b92c-4b83-82d4-58ca27c49769 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:17.901202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:17.907027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5331e4-9286-4f8b-9dce-1586519340a7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:17.908430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479d6fc-fc1b-4f8e-b890-3b1be6187b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:17.909380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479d6fc-fc1b-4f8e-b890-3b1be6187b20 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:17.909914 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479d6fc-fc1b-4f8e-b890-3b1be6187b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:17.910409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479d6fc-fc1b-4f8e-b890-3b1be6187b20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:17.920928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479d6fc-fc1b-4f8e-b890-3b1be6187b20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-e479d6fc-fc1b-4f8e-b890-3b1be6187b20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:17.929449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:17.930310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:17.931074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:17.933372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 112/213] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:17.943365 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479d6fc-fc1b-4f8e-b890-3b1be6187b20 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:17.944752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c82ae3-c6bc-429d-9591-1cd4bf73c14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:17.945571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c82ae3-c6bc-429d-9591-1cd4bf73c14e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:17.945850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c82ae3-c6bc-429d-9591-1cd4bf73c14e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:17.946121 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c82ae3-c6bc-429d-9591-1cd4bf73c14e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:17.947275 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c82ae3-c6bc-429d-9591-1cd4bf73c14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:17.948917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c82ae3-c6bc-429d-9591-1cd4bf73c14e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:17.955118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 102/214] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:17.956100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c82ae3-c6bc-429d-9591-1cd4bf73c14e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:17.991168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:18.050012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479d6fc-fc1b-4f8e-b890-3b1be6187b20 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:18.060165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c82ae3-c6bc-429d-9591-1cd4bf73c14e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:18.086336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a891a4-61f2-479c-8e6f-71b6e0d2e129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:18.087334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a891a4-61f2-479c-8e6f-71b6e0d2e129 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.087841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a891a4-61f2-479c-8e6f-71b6e0d2e129 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.088305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a891a4-61f2-479c-8e6f-71b6e0d2e129 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.089280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a891a4-61f2-479c-8e6f-71b6e0d2e129 None admin] RBAC: Adding query filter params (name=member) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:18.089780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a891a4-61f2-479c-8e6f-71b6e0d2e129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:18.091371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a891a4-61f2-479c-8e6f-71b6e0d2e129 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:18.096676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884badf3-2875-45cd-b4d7-72a58b02a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:18.097569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884badf3-2875-45cd-b4d7-72a58b02a52c None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.098058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884badf3-2875-45cd-b4d7-72a58b02a52c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.099063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884badf3-2875-45cd-b4d7-72a58b02a52c None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.100149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884badf3-2875-45cd-b4d7-72a58b02a52c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:18.101440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 113/215] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:18.102311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884badf3-2875-45cd-b4d7-72a58b02a52c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:18.107084 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a891a4-61f2-479c-8e6f-71b6e0d2e129 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:18.150115 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:18.215118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a891a4-61f2-479c-8e6f-71b6e0d2e129 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:18.220895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 103/216] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:17 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:18.259187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba5ff6-004d-4c61-8d63-be1c0aa555c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:18.260223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba5ff6-004d-4c61-8d63-be1c0aa555c3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.260503 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba5ff6-004d-4c61-8d63-be1c0aa555c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.260795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba5ff6-004d-4c61-8d63-be1c0aa555c3 None admin] PATH_INFO: `/v3/users/7ae4a2043a0c463b8188ed634893d10c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.280881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba5ff6-004d-4c61-8d63-be1c0aa555c3 None admin] RBAC: Authorizing `identity:get_user(user_id=7ae4a2043a0c463b8188ed634893d10c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:18.282364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba5ff6-004d-4c61-8d63-be1c0aa555c3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:18.284479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 114/217] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:18 2022] GET /identity/v3/users/7ae4a2043a0c463b8188ed634893d10c => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:18.290932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884badf3-2875-45cd-b4d7-72a58b02a52c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:18.399536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:18.408225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884badf3-2875-45cd-b4d7-72a58b02a52c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:18.443954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f31df-0d92-428c-80fb-296de8451023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:18.444616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f31df-0d92-428c-80fb-296de8451023 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.444943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f31df-0d92-428c-80fb-296de8451023 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.445266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f31df-0d92-428c-80fb-296de8451023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.518924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f31df-0d92-428c-80fb-296de8451023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:18.520324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f31df-0d92-428c-80fb-296de8451023 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:18.559129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c063354-f6db-4357-b273-1666ec068d84 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.559393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c063354-f6db-4357-b273-1666ec068d84 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.559642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c063354-f6db-4357-b273-1666ec068d84 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.562226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 115/218] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:50:18.567498 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e13408-9cbe-409c-8cad-0341429adc74 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.567791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e13408-9cbe-409c-8cad-0341429adc74 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.568075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e13408-9cbe-409c-8cad-0341429adc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.589116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-60e13408-9cbe-409c-8cad-0341429adc74 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:18.590327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-60e13408-9cbe-409c-8cad-0341429adc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:18.626869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:18.636694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 104/219] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:18.642375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f31df-0d92-428c-80fb-296de8451023 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:18.670572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 116/220] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:18.708718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e13408-9cbe-409c-8cad-0341429adc74 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:18.746812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:18.770209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f31df-0d92-428c-80fb-296de8451023 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:18.816026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b8d32-5842-4274-bee3-07f90683866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:18.816913 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b8d32-5842-4274-bee3-07f90683866b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.817547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b8d32-5842-4274-bee3-07f90683866b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.817827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b8d32-5842-4274-bee3-07f90683866b None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.819320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b8d32-5842-4274-bee3-07f90683866b None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:18.821525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b8d32-5842-4274-bee3-07f90683866b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:18.823253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e13408-9cbe-409c-8cad-0341429adc74 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:18.823535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 105/221] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:18 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:18.824735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c74a7-de9e-4399-91ee-339d9dd77cca None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.825047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c74a7-de9e-4399-91ee-339d9dd77cca None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.830799 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c74a7-de9e-4399-91ee-339d9dd77cca None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.831875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 106/222] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:18.834436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c74a7-de9e-4399-91ee-339d9dd77cca None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:18.869570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343ce02-643e-4798-ba35-061ee09bfeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:18.870540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343ce02-643e-4798-ba35-061ee09bfeac None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.870973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343ce02-643e-4798-ba35-061ee09bfeac None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.871350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343ce02-643e-4798-ba35-061ee09bfeac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.891444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343ce02-643e-4798-ba35-061ee09bfeac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-0343ce02-643e-4798-ba35-061ee09bfeac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:18.902235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:18.903095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:18.903863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:18.904531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 117/223] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:50:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:18.905590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:18.907317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d851db-2f0c-4dc7-b495-feef1603d40c None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.907976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d851db-2f0c-4dc7-b495-feef1603d40c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.908396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d851db-2f0c-4dc7-b495-feef1603d40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.929184 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b9d851db-2f0c-4dc7-b495-feef1603d40c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:18.931157 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b9d851db-2f0c-4dc7-b495-feef1603d40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:18.952287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c74a7-de9e-4399-91ee-339d9dd77cca None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:18.996758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a287da-3ade-4d33-8d13-71841deda2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:18.997482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a287da-3ade-4d33-8d13-71841deda2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:18.997776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a287da-3ade-4d33-8d13-71841deda2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:18.998052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a287da-3ade-4d33-8d13-71841deda2b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:18.998900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a287da-3ade-4d33-8d13-71841deda2b3 None admin] RBAC: Adding query filter params (role.id=61e62f29eafd45c8b9fb99205a60bc4e, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b, user.id=7ae4a2043a0c463b8188ed634893d10c) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:18.999215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a287da-3ade-4d33-8d13-71841deda2b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:19.000674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a287da-3ade-4d33-8d13-71841deda2b3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:19.009945 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 107/224] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:18 2022] GET /identity/v3/role_assignments?user.id=7ae4a2043a0c463b8188ed634893d10c&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:19.011210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a287da-3ade-4d33-8d13-71841deda2b3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:19.025829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 118/225] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:19.108849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d851db-2f0c-4dc7-b495-feef1603d40c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:19.131780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:19.195133 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a287da-3ade-4d33-8d13-71841deda2b3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:19.225128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:19.233263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb297b6c-8149-4f86-a698-b1de210fa9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:19.234211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb297b6c-8149-4f86-a698-b1de210fa9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:19.234688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb297b6c-8149-4f86-a698-b1de210fa9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:19.235131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb297b6c-8149-4f86-a698-b1de210fa9d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:19.236084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb297b6c-8149-4f86-a698-b1de210fa9d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:19.236562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb297b6c-8149-4f86-a698-b1de210fa9d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:19.238262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb297b6c-8149-4f86-a698-b1de210fa9d4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:19.247808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 108/226] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:19.253345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb297b6c-8149-4f86-a698-b1de210fa9d4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:19.353134 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d851db-2f0c-4dc7-b495-feef1603d40c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:19.362398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb297b6c-8149-4f86-a698-b1de210fa9d4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:19.398696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81553802-e837-4200-b0a6-3311f816c065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:19.399936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553802-e837-4200-b0a6-3311f816c065 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:19.400421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553802-e837-4200-b0a6-3311f816c065 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:19.400934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553802-e837-4200-b0a6-3311f816c065 None admin] PATH_INFO: `/v3/users/17ab4b315fb545808d2644766f27d8fb` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:19.406525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81553802-e837-4200-b0a6-3311f816c065 None admin] RBAC: Authorizing `identity:get_user(user_id=17ab4b315fb545808d2644766f27d8fb)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:19.408268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81553802-e837-4200-b0a6-3311f816c065 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:19.412480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 109/227] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:19 2022] GET /identity/v3/users/17ab4b315fb545808d2644766f27d8fb => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:19.418492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81553802-e837-4200-b0a6-3311f816c065 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:19.425225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c850a-cfef-4ea7-823b-45ed043d8ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:19.426278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c850a-cfef-4ea7-823b-45ed043d8ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:19.429051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c850a-cfef-4ea7-823b-45ed043d8ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:19.430411 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c850a-cfef-4ea7-823b-45ed043d8ec6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:19.452660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c850a-cfef-4ea7-823b-45ed043d8ec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-1b8c850a-cfef-4ea7-823b-45ed043d8ec6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:19.467037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:19.467930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:19.468697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:19.478419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 119/228] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:19.502084 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c850a-cfef-4ea7-823b-45ed043d8ec6 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:19.529111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:19.611780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81553802-e837-4200-b0a6-3311f816c065 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:19.649439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:19.678244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15384028-56cc-4190-8589-0b5769164c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:19.679119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15384028-56cc-4190-8589-0b5769164c11 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:19.679405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15384028-56cc-4190-8589-0b5769164c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:19.679683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15384028-56cc-4190-8589-0b5769164c11 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:19.681149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15384028-56cc-4190-8589-0b5769164c11 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:19.682622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15384028-56cc-4190-8589-0b5769164c11 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:19.684846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 110/229] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:19 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:19.690249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15384028-56cc-4190-8589-0b5769164c11 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:19.695233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c850a-cfef-4ea7-823b-45ed043d8ec6 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:19.763412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396b072-a0a9-484d-a594-50b69f109067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:19.764092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396b072-a0a9-484d-a594-50b69f109067 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:19.764379 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396b072-a0a9-484d-a594-50b69f109067 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:19.764658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396b072-a0a9-484d-a594-50b69f109067 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:19.765448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396b072-a0a9-484d-a594-50b69f109067 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:19.765756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396b072-a0a9-484d-a594-50b69f109067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:19.767160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396b072-a0a9-484d-a594-50b69f109067 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:19.788196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 120/230] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:19.796119 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396b072-a0a9-484d-a594-50b69f109067 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:19.909996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15384028-56cc-4190-8589-0b5769164c11 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:19.940575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396b072-a0a9-484d-a594-50b69f109067 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:19.971707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:19.976244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6a698-3708-47e4-bc5e-0a2b515158ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:19.977411 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6a698-3708-47e4-bc5e-0a2b515158ab None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:19.977698 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6a698-3708-47e4-bc5e-0a2b515158ab None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:19.977977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6a698-3708-47e4-bc5e-0a2b515158ab None admin] PATH_INFO: `/v3/users/17ab4b315fb545808d2644766f27d8fb` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:19.979268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6a698-3708-47e4-bc5e-0a2b515158ab None admin] RBAC: Authorizing `identity:get_user(user_id=17ab4b315fb545808d2644766f27d8fb)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:19.983164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6a698-3708-47e4-bc5e-0a2b515158ab None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:19.987108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 121/231] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:19 2022] GET /identity/v3/users/17ab4b315fb545808d2644766f27d8fb => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:19.988316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6a698-3708-47e4-bc5e-0a2b515158ab None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:20.017111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:20.021436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7588b-8e6f-44d4-aa27-d9385a6a1141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:20.022423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7588b-8e6f-44d4-aa27-d9385a6a1141 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:20.022788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7588b-8e6f-44d4-aa27-d9385a6a1141 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:20.023145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7588b-8e6f-44d4-aa27-d9385a6a1141 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/users/17ab4b315fb545808d2644766f27d8fb/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:20.029557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7588b-8e6f-44d4-aa27-d9385a6a1141 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a83413aa348d7a1e9fcd9b258330b, user_id=17ab4b315fb545808d2644766f27d8fb, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:20.031147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7588b-8e6f-44d4-aa27-d9385a6a1141 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:20.098132 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6a698-3708-47e4-bc5e-0a2b515158ab None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:20.136391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56130dce-5ab8-40d8-a47a-0fdc5aa8340f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:20.137536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56130dce-5ab8-40d8-a47a-0fdc5aa8340f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:20.138003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56130dce-5ab8-40d8-a47a-0fdc5aa8340f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:20.138556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56130dce-5ab8-40d8-a47a-0fdc5aa8340f None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:20.139936 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56130dce-5ab8-40d8-a47a-0fdc5aa8340f None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:20.141612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56130dce-5ab8-40d8-a47a-0fdc5aa8340f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:20.148392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 122/232] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:19 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:20.149971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56130dce-5ab8-40d8-a47a-0fdc5aa8340f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:20.166508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 111/233] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:50:19 2022] PUT /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/users/17ab4b315fb545808d2644766f27d8fb/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:20.247165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:20.275321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56130dce-5ab8-40d8-a47a-0fdc5aa8340f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:20.315016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916cec5d-36bb-4de4-9331-3a7af0246d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:20.315941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cec5d-36bb-4de4-9331-3a7af0246d08 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:20.316224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cec5d-36bb-4de4-9331-3a7af0246d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:20.316501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cec5d-36bb-4de4-9331-3a7af0246d08 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/users/17ab4b315fb545808d2644766f27d8fb/roles/2991af796dcd45e2b86208339339b977` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:20.319526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916cec5d-36bb-4de4-9331-3a7af0246d08 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a83413aa348d7a1e9fcd9b258330b, user_id=17ab4b315fb545808d2644766f27d8fb, role_id=2991af796dcd45e2b86208339339b977)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:20.321052 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916cec5d-36bb-4de4-9331-3a7af0246d08 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:20.449050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 123/234] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:50:20 2022] PUT /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/users/17ab4b315fb545808d2644766f27d8fb/roles/2991af796dcd45e2b86208339339b977 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:50:20.707712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c31a0-418a-402a-9510-0c242697efaa None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:20.708800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c31a0-418a-402a-9510-0c242697efaa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:20.708800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c31a0-418a-402a-9510-0c242697efaa None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:20.710303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 112/235] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:20.715647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5789590-2eff-4e3e-8b8c-193e0e2e042c None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:20.716067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5789590-2eff-4e3e-8b8c-193e0e2e042c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:20.716265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5789590-2eff-4e3e-8b8c-193e0e2e042c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:20.741797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f5789590-2eff-4e3e-8b8c-193e0e2e042c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:20.743021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f5789590-2eff-4e3e-8b8c-193e0e2e042c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:20.818340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 124/236] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:20.825122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4aaa5-39e2-442d-ac2a-ed3df17a30a1 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:20.825422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4aaa5-39e2-442d-ac2a-ed3df17a30a1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:20.825708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4aaa5-39e2-442d-ac2a-ed3df17a30a1 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:20.828288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 113/237] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:20.832672 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76844d4b-3ce4-4c8a-aa16-61fef7dd0f25 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:20.833034 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76844d4b-3ce4-4c8a-aa16-61fef7dd0f25 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:20.833323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76844d4b-3ce4-4c8a-aa16-61fef7dd0f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:20.844141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4aaa5-39e2-442d-ac2a-ed3df17a30a1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:20.850427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-76844d4b-3ce4-4c8a-aa16-61fef7dd0f25 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:20.852206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-76844d4b-3ce4-4c8a-aa16-61fef7dd0f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:20.939307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:20.951197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 125/238] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:21.001129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76844d4b-3ce4-4c8a-aa16-61fef7dd0f25 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:21.011261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4aaa5-39e2-442d-ac2a-ed3df17a30a1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:21.045764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9402b83-afe6-4147-bd5d-b464e114e105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:21.046915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9402b83-afe6-4147-bd5d-b464e114e105 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:21.047289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9402b83-afe6-4147-bd5d-b464e114e105 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:21.047656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9402b83-afe6-4147-bd5d-b464e114e105 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:21.058731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9402b83-afe6-4147-bd5d-b464e114e105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-d9402b83-afe6-4147-bd5d-b464e114e105 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:21.066508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:21.067983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:21.068739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:21.069505 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 114/239] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:21.081808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9402b83-afe6-4147-bd5d-b464e114e105 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:21.141356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:21.176789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76844d4b-3ce4-4c8a-aa16-61fef7dd0f25 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:21.199295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9402b83-afe6-4147-bd5d-b464e114e105 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:21.229457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7f285-aeae-49c2-aeee-4b681d3ca44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:21.230534 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7f285-aeae-49c2-aeee-4b681d3ca44a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:21.230957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7f285-aeae-49c2-aeee-4b681d3ca44a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:21.231369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7f285-aeae-49c2-aeee-4b681d3ca44a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:21.232706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7f285-aeae-49c2-aeee-4b681d3ca44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:21.234411 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7f285-aeae-49c2-aeee-4b681d3ca44a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:21.236556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 126/240] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:21.242531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7f285-aeae-49c2-aeee-4b681d3ca44a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:21.246584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7947c9-0378-48b2-b4b9-7095548ef99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:21.246584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7947c9-0378-48b2-b4b9-7095548ef99e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:21.246584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7947c9-0378-48b2-b4b9-7095548ef99e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:21.246584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7947c9-0378-48b2-b4b9-7095548ef99e None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:21.246908 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7947c9-0378-48b2-b4b9-7095548ef99e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:21.246908 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7947c9-0378-48b2-b4b9-7095548ef99e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:21.247865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7947c9-0378-48b2-b4b9-7095548ef99e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:21.257438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 115/241] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:21.262648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7947c9-0378-48b2-b4b9-7095548ef99e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:21.356619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:21.424501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7f285-aeae-49c2-aeee-4b681d3ca44a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:21.431313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7947c9-0378-48b2-b4b9-7095548ef99e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:21.432913 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:21.473785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4c06c-fe69-441f-af76-791a88dd83c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:21.474457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4c06c-fe69-441f-af76-791a88dd83c9 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:21.474740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4c06c-fe69-441f-af76-791a88dd83c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:21.474882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e564e4-3929-4037-99e4-7ce70cd1efd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:21.475012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4c06c-fe69-441f-af76-791a88dd83c9 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:21.475712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4c06c-fe69-441f-af76-791a88dd83c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:21.475924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e564e4-3929-4037-99e4-7ce70cd1efd7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:21.476278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e564e4-3929-4037-99e4-7ce70cd1efd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:21.476919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e564e4-3929-4037-99e4-7ce70cd1efd7 None admin] PATH_INFO: `/v3/users/ea81b012b9064e21b61c0813b08f4e11` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:21.477437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4c06c-fe69-441f-af76-791a88dd83c9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:21.495339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e564e4-3929-4037-99e4-7ce70cd1efd7 None admin] RBAC: Authorizing `identity:get_user(user_id=ea81b012b9064e21b61c0813b08f4e11)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:21.496924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e564e4-3929-4037-99e4-7ce70cd1efd7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:21.498937 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 116/242] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:21 2022] GET /identity/v3/users/ea81b012b9064e21b61c0813b08f4e11 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:21.503495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ede280-695b-4178-8579-6d10114fb5d3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:21.503495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ede280-695b-4178-8579-6d10114fb5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:21.503495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ede280-695b-4178-8579-6d10114fb5d3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:21.503495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 117/243] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:21.509832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ede280-695b-4178-8579-6d10114fb5d3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:21.601232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 127/244] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:21 2022] POST /identity/v3/users => generated 293 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:21.603528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049224b-7ec4-4765-bbef-a37b74b65ae9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:21.604403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049224b-7ec4-4765-bbef-a37b74b65ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:21.605462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049224b-7ec4-4765-bbef-a37b74b65ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:21.627265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3049224b-7ec4-4765-bbef-a37b74b65ae9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:21.628813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ede280-695b-4178-8579-6d10114fb5d3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:21.629302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3049224b-7ec4-4765-bbef-a37b74b65ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:21.702099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e5603-ee59-445e-b4ba-01ac81407d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:21.702786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e5603-ee59-445e-b4ba-01ac81407d8b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:21.703074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e5603-ee59-445e-b4ba-01ac81407d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:21.703353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e5603-ee59-445e-b4ba-01ac81407d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:21.717958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 128/245] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:21.758195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:21.759078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049224b-7ec4-4765-bbef-a37b74b65ae9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:21.799539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e5603-ee59-445e-b4ba-01ac81407d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:21.800977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e5603-ee59-445e-b4ba-01ac81407d8b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:21.820194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:21.886582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049224b-7ec4-4765-bbef-a37b74b65ae9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:21.921832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 118/246] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:21.927672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e5603-ee59-445e-b4ba-01ac81407d8b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:21.934012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74229f-b69a-434e-ad7a-7cf8f216309f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:21.934816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74229f-b69a-434e-ad7a-7cf8f216309f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:21.935098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74229f-b69a-434e-ad7a-7cf8f216309f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:21.935376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74229f-b69a-434e-ad7a-7cf8f216309f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:21.953442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74229f-b69a-434e-ad7a-7cf8f216309f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-ee74229f-b69a-434e-ad7a-7cf8f216309f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:21.963673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:21.964666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:21.965626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 129/247] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:21.971094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74229f-b69a-434e-ad7a-7cf8f216309f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:22.023829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:22.072300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e5603-ee59-445e-b4ba-01ac81407d8b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:22.088695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74229f-b69a-434e-ad7a-7cf8f216309f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:22.100795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:22.112636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf54000-fa7e-4c5b-a29d-91c1ac6bf1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:22.114492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf54000-fa7e-4c5b-a29d-91c1ac6bf1cf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:22.114492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf54000-fa7e-4c5b-a29d-91c1ac6bf1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:22.114492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf54000-fa7e-4c5b-a29d-91c1ac6bf1cf None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:22.116053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf54000-fa7e-4c5b-a29d-91c1ac6bf1cf None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:22.117812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf54000-fa7e-4c5b-a29d-91c1ac6bf1cf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:22.120033 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 119/248] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:21 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:22.126150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf54000-fa7e-4c5b-a29d-91c1ac6bf1cf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:22.133989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc233ed7-3759-4e4f-9a60-0d2ed82cc885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:22.134914 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc233ed7-3759-4e4f-9a60-0d2ed82cc885 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:22.135408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc233ed7-3759-4e4f-9a60-0d2ed82cc885 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:22.136326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc233ed7-3759-4e4f-9a60-0d2ed82cc885 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:22.137316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc233ed7-3759-4e4f-9a60-0d2ed82cc885 None admin] RBAC: Adding query filter params (name=member) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:22.138182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc233ed7-3759-4e4f-9a60-0d2ed82cc885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:22.141375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc233ed7-3759-4e4f-9a60-0d2ed82cc885 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:22.158645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 130/249] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:22.169709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc233ed7-3759-4e4f-9a60-0d2ed82cc885 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:22.254370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf54000-fa7e-4c5b-a29d-91c1ac6bf1cf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:22.291599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:22.301161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc233ed7-3759-4e4f-9a60-0d2ed82cc885 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:22.303370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab0a46-e7a0-483c-9beb-fabd38bd55a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:22.303489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab0a46-e7a0-483c-9beb-fabd38bd55a8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:22.303720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab0a46-e7a0-483c-9beb-fabd38bd55a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:22.304089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab0a46-e7a0-483c-9beb-fabd38bd55a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:22.305006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab0a46-e7a0-483c-9beb-fabd38bd55a8 None admin] RBAC: Adding query filter params (role.id=583fe587d8d94b0fad411aeefb4181e0, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b, user.id=ea81b012b9064e21b61c0813b08f4e11) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:22.305323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab0a46-e7a0-483c-9beb-fabd38bd55a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:22.306819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab0a46-e7a0-483c-9beb-fabd38bd55a8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:22.313174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 120/250] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:22 2022] GET /identity/v3/role_assignments?user.id=ea81b012b9064e21b61c0813b08f4e11&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=583fe587d8d94b0fad411aeefb4181e0 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:22.389718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cc0c0-c376-48f9-be47-7138a8fd5894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:22.390561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cc0c0-c376-48f9-be47-7138a8fd5894 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:22.390849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cc0c0-c376-48f9-be47-7138a8fd5894 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:22.391129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cc0c0-c376-48f9-be47-7138a8fd5894 None admin] PATH_INFO: `/v3/users/7ae4a2043a0c463b8188ed634893d10c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:22.392474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cc0c0-c376-48f9-be47-7138a8fd5894 None admin] RBAC: Authorizing `identity:get_user(user_id=7ae4a2043a0c463b8188ed634893d10c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:22.393980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cc0c0-c376-48f9-be47-7138a8fd5894 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:22.396004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 131/251] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:22 2022] GET /identity/v3/users/7ae4a2043a0c463b8188ed634893d10c => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:22.401582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab0a46-e7a0-483c-9beb-fabd38bd55a8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:22.536162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab0a46-e7a0-483c-9beb-fabd38bd55a8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:22.576206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87bf67a-5310-4823-8c6e-86a4561507d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:22.578612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bf67a-5310-4823-8c6e-86a4561507d9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:22.578612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bf67a-5310-4823-8c6e-86a4561507d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:22.578794 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87bf67a-5310-4823-8c6e-86a4561507d9 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:22.581485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87bf67a-5310-4823-8c6e-86a4561507d9 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:22.583685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87bf67a-5310-4823-8c6e-86a4561507d9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:22.586003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 121/252] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:22 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:22.591373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cc0c0-c376-48f9-be47-7138a8fd5894 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:22.661967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:22.731041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cc0c0-c376-48f9-be47-7138a8fd5894 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:22.758128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a074d48-4261-48ae-b884-c6b3de247eeb None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:22.758441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a074d48-4261-48ae-b884-c6b3de247eeb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:22.758951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a074d48-4261-48ae-b884-c6b3de247eeb None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:22.760310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 122/253] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:22.765192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bcdad-e90e-45eb-a13f-37a210d39ff2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:22.765507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bcdad-e90e-45eb-a13f-37a210d39ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:22.765763 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bcdad-e90e-45eb-a13f-37a210d39ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:22.773344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e99e1-b910-45d1-abe6-9b0e8e42aa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:22.774251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e99e1-b910-45d1-abe6-9b0e8e42aa34 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:22.774540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e99e1-b910-45d1-abe6-9b0e8e42aa34 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:22.774818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e99e1-b910-45d1-abe6-9b0e8e42aa34 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/users/7ae4a2043a0c463b8188ed634893d10c/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:22.782217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e99e1-b910-45d1-abe6-9b0e8e42aa34 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a83413aa348d7a1e9fcd9b258330b, user_id=7ae4a2043a0c463b8188ed634893d10c, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:22.783373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6e7bcdad-e90e-45eb-a13f-37a210d39ff2 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:22.783827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e99e1-b910-45d1-abe6-9b0e8e42aa34 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:22.784565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6e7bcdad-e90e-45eb-a13f-37a210d39ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:22.844234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:22.856677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 123/254] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:22.869802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 132/255] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:50:22 2022] PUT /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/users/7ae4a2043a0c463b8188ed634893d10c/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:22.880993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bcdad-e90e-45eb-a13f-37a210d39ff2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:23.041119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bcdad-e90e-45eb-a13f-37a210d39ff2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:23.087928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3cbbe-2566-4b7e-bd19-08de3d7e1709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:23.088751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3cbbe-2566-4b7e-bd19-08de3d7e1709 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:23.089072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3cbbe-2566-4b7e-bd19-08de3d7e1709 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:23.089348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3cbbe-2566-4b7e-bd19-08de3d7e1709 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:23.107055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3cbbe-2566-4b7e-bd19-08de3d7e1709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a0c3cbbe-2566-4b7e-bd19-08de3d7e1709 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:23.114958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:23.115891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:23.116675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 124/256] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:50:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:23.125917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e99e1-b910-45d1-abe6-9b0e8e42aa34 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:23.255636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:23.259277 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e99e1-b910-45d1-abe6-9b0e8e42aa34 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:23.303475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d41cda-75fe-488d-b098-6772c9de6163 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:23.305230 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d41cda-75fe-488d-b098-6772c9de6163 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:23.306409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d41cda-75fe-488d-b098-6772c9de6163 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:23.308534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 125/257] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:23.314307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193e1bc-0087-4678-ac80-b8e6f10d5167 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:23.316570 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193e1bc-0087-4678-ac80-b8e6f10d5167 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:23.317447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193e1bc-0087-4678-ac80-b8e6f10d5167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:23.320103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cec3d4-aa1e-456e-9a98-98faaafc8a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:23.321021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cec3d4-aa1e-456e-9a98-98faaafc8a09 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:23.321842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cec3d4-aa1e-456e-9a98-98faaafc8a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:23.322795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cec3d4-aa1e-456e-9a98-98faaafc8a09 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:23.324804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cec3d4-aa1e-456e-9a98-98faaafc8a09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:23.325315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cec3d4-aa1e-456e-9a98-98faaafc8a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:23.328164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cec3d4-aa1e-456e-9a98-98faaafc8a09 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:23.349255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 133/258] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:23.350568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c193e1bc-0087-4678-ac80-b8e6f10d5167 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:23.352202 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c193e1bc-0087-4678-ac80-b8e6f10d5167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:23.374206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cec3d4-aa1e-456e-9a98-98faaafc8a09 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:23.453120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:23.499153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 126/259] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:23.563634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cec3d4-aa1e-456e-9a98-98faaafc8a09 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:23.565660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193e1bc-0087-4678-ac80-b8e6f10d5167 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:23.646249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaec8c2-637d-4e68-8d61-3d75b23d6054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:23.647128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaec8c2-637d-4e68-8d61-3d75b23d6054 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:23.647415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaec8c2-637d-4e68-8d61-3d75b23d6054 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:23.647689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaec8c2-637d-4e68-8d61-3d75b23d6054 None admin] PATH_INFO: `/v3/users/17ab4b315fb545808d2644766f27d8fb` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:23.650365 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaec8c2-637d-4e68-8d61-3d75b23d6054 None admin] RBAC: Authorizing `identity:get_user(user_id=17ab4b315fb545808d2644766f27d8fb)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:23.652893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaec8c2-637d-4e68-8d61-3d75b23d6054 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:23.654708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 134/260] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:23 2022] GET /identity/v3/users/17ab4b315fb545808d2644766f27d8fb => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:23.671390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaec8c2-637d-4e68-8d61-3d75b23d6054 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:23.745068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193e1bc-0087-4678-ac80-b8e6f10d5167 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:23.752264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:23.799197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaec8c2-637d-4e68-8d61-3d75b23d6054 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:23.811141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e72dde-6c31-40ab-9a23-71875a1d15ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:23.811975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e72dde-6c31-40ab-9a23-71875a1d15ba None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:23.812264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e72dde-6c31-40ab-9a23-71875a1d15ba None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:23.812597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e72dde-6c31-40ab-9a23-71875a1d15ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:23.835335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e72dde-6c31-40ab-9a23-71875a1d15ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-85e72dde-6c31-40ab-9a23-71875a1d15ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:23.843785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:23.844730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:23.845548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 127/261] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:23.851735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e72dde-6c31-40ab-9a23-71875a1d15ba None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:23.866602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15e77f-ae9f-42bf-a4c6-386dd46bbb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:23.867271 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15e77f-ae9f-42bf-a4c6-386dd46bbb98 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:23.867559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15e77f-ae9f-42bf-a4c6-386dd46bbb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:23.867837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15e77f-ae9f-42bf-a4c6-386dd46bbb98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:23.917108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:23.940920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15e77f-ae9f-42bf-a4c6-386dd46bbb98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:23.940920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15e77f-ae9f-42bf-a4c6-386dd46bbb98 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:23.975685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:24.016394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e72dde-6c31-40ab-9a23-71875a1d15ba None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:24.043923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 135/262] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:50:24.049757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15e77f-ae9f-42bf-a4c6-386dd46bbb98 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:24.073988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03d531-5039-440c-af67-138708396188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:24.075026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03d531-5039-440c-af67-138708396188 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:24.075505 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03d531-5039-440c-af67-138708396188 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:24.075972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03d531-5039-440c-af67-138708396188 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:24.076991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03d531-5039-440c-af67-138708396188 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:24.077495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03d531-5039-440c-af67-138708396188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:24.079093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03d531-5039-440c-af67-138708396188 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:24.088729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 128/263] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:24.098180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03d531-5039-440c-af67-138708396188 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:24.235861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15e77f-ae9f-42bf-a4c6-386dd46bbb98 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:24.266654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:24.277137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92eed98-8b51-44e2-8cd8-6401cdd1442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:24.278346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92eed98-8b51-44e2-8cd8-6401cdd1442d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:24.278637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92eed98-8b51-44e2-8cd8-6401cdd1442d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:24.278916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92eed98-8b51-44e2-8cd8-6401cdd1442d None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:24.281921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92eed98-8b51-44e2-8cd8-6401cdd1442d None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:24.283403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92eed98-8b51-44e2-8cd8-6401cdd1442d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:24.285456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 136/264] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:24 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:24.294430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92eed98-8b51-44e2-8cd8-6401cdd1442d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:24.331827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03d531-5039-440c-af67-138708396188 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:24.337062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:24.414610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ddc98-c655-4809-b683-c0419f6f960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:24.415553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ddc98-c655-4809-b683-c0419f6f960e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:24.415833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ddc98-c655-4809-b683-c0419f6f960e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:24.416107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ddc98-c655-4809-b683-c0419f6f960e None admin] PATH_INFO: `/v3/users/17ab4b315fb545808d2644766f27d8fb` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:24.418213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ddc98-c655-4809-b683-c0419f6f960e None admin] RBAC: Authorizing `identity:get_user(user_id=17ab4b315fb545808d2644766f27d8fb)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:24.419676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ddc98-c655-4809-b683-c0419f6f960e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:24.421333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 129/265] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:24 2022] GET /identity/v3/users/17ab4b315fb545808d2644766f27d8fb => generated 283 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:24.427532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ddc98-c655-4809-b683-c0419f6f960e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:24.464180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92eed98-8b51-44e2-8cd8-6401cdd1442d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:24.543333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878b4e5-3d89-4af8-9bf0-8199eafce9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:24.544004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878b4e5-3d89-4af8-9bf0-8199eafce9fb None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:24.544291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878b4e5-3d89-4af8-9bf0-8199eafce9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:24.544567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878b4e5-3d89-4af8-9bf0-8199eafce9fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:24.545438 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878b4e5-3d89-4af8-9bf0-8199eafce9fb None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b, user.id=17ab4b315fb545808d2644766f27d8fb) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:24.545753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878b4e5-3d89-4af8-9bf0-8199eafce9fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:24.547234 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878b4e5-3d89-4af8-9bf0-8199eafce9fb None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:24.558001 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 137/266] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:24 2022] GET /identity/v3/role_assignments?user.id=17ab4b315fb545808d2644766f27d8fb&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:24.591130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ddc98-c655-4809-b683-c0419f6f960e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:24.648706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1a174-0fac-49f1-a400-80870d8b881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:24.649396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1a174-0fac-49f1-a400-80870d8b881c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:24.649681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1a174-0fac-49f1-a400-80870d8b881c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:24.650023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1a174-0fac-49f1-a400-80870d8b881c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:24.701329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:24.729518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1a174-0fac-49f1-a400-80870d8b881c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:24.731004 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1a174-0fac-49f1-a400-80870d8b881c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:24.850706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 130/267] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:50:24.856508 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878b4e5-3d89-4af8-9bf0-8199eafce9fb None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:24.920006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:24.955203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025e58-9b00-4603-a042-1e9241cdbebf None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:24.955441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025e58-9b00-4603-a042-1e9241cdbebf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:24.955689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025e58-9b00-4603-a042-1e9241cdbebf None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:24.957658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 131/268] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:24.966248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878b4e5-3d89-4af8-9bf0-8199eafce9fb None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:24.967203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7868-ebb1-4840-845c-4c153a7b3439 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:24.967498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7868-ebb1-4840-845c-4c153a7b3439 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:24.967780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7868-ebb1-4840-845c-4c153a7b3439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:24.990208 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e32c7868-ebb1-4840-845c-4c153a7b3439 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:24.992532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e32c7868-ebb1-4840-845c-4c153a7b3439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:25.009700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f64c8-e890-40eb-ba62-0f5c21dc00f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:25.011176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f64c8-e890-40eb-ba62-0f5c21dc00f0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.011645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f64c8-e890-40eb-ba62-0f5c21dc00f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.012227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f64c8-e890-40eb-ba62-0f5c21dc00f0 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.014553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f64c8-e890-40eb-ba62-0f5c21dc00f0 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:25.016017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f64c8-e890-40eb-ba62-0f5c21dc00f0 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:25.018105 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 138/269] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:24 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:25.025079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f64c8-e890-40eb-ba62-0f5c21dc00f0 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:25.085176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:25.092069 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 132/270] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:25.120454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7868-ebb1-4840-845c-4c153a7b3439 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:25.140370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f64c8-e890-40eb-ba62-0f5c21dc00f0 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:25.218299 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:25.238537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7868-ebb1-4840-845c-4c153a7b3439 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:25.245235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839df3cc-d559-4edb-8597-803a03e8cd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:25.245905 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df3cc-d559-4edb-8597-803a03e8cd03 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.246190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df3cc-d559-4edb-8597-803a03e8cd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.246467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df3cc-d559-4edb-8597-803a03e8cd03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.247309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df3cc-d559-4edb-8597-803a03e8cd03 None admin] RBAC: Adding query filter params (role.id=2991af796dcd45e2b86208339339b977, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b, user.id=17ab4b315fb545808d2644766f27d8fb) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:25.247678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df3cc-d559-4edb-8597-803a03e8cd03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:25.249200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df3cc-d559-4edb-8597-803a03e8cd03 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:25.257036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 139/271] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:25 2022] GET /identity/v3/role_assignments?user.id=17ab4b315fb545808d2644766f27d8fb&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=2991af796dcd45e2b86208339339b977 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:25.258089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849fc72-e0e6-49c5-a3a3-87d2423a90fe None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.258377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849fc72-e0e6-49c5-a3a3-87d2423a90fe None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.258652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849fc72-e0e6-49c5-a3a3-87d2423a90fe None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.260074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 140/272] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:25.270532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed7bb-0d32-4a7f-b690-be223026f01a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.270850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed7bb-0d32-4a7f-b690-be223026f01a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.271095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed7bb-0d32-4a7f-b690-be223026f01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.280078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ea7dc-d846-4dae-854e-18cd265821d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:25.280939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ea7dc-d846-4dae-854e-18cd265821d3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.281233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ea7dc-d846-4dae-854e-18cd265821d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.281522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ea7dc-d846-4dae-854e-18cd265821d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.282769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ea7dc-d846-4dae-854e-18cd265821d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:25.284254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ea7dc-d846-4dae-854e-18cd265821d3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:25.286771 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 133/273] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:25.292424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ea7dc-d846-4dae-854e-18cd265821d3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:25.296213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-a7aed7bb-0d32-4a7f-b690-be223026f01a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:25.300681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a7aed7bb-0d32-4a7f-b690-be223026f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:25.410917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ea7dc-d846-4dae-854e-18cd265821d3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:25.425295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:25.454846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f37a9-f13b-42b3-ad19-7bbde3c98a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:25.455568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f37a9-f13b-42b3-ad19-7bbde3c98a89 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.455854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f37a9-f13b-42b3-ad19-7bbde3c98a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.456138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f37a9-f13b-42b3-ad19-7bbde3c98a89 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.459011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f37a9-f13b-42b3-ad19-7bbde3c98a89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:25.460552 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f37a9-f13b-42b3-ad19-7bbde3c98a89 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:25.483188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 141/274] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:25.508239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aed7bb-0d32-4a7f-b690-be223026f01a None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:25.585743 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 134/275] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:25 2022] POST /identity/v3/users => generated 293 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:50:25.588965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7ed1a-94f9-41e1-a974-ab789a31f745 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.589263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7ed1a-94f9-41e1-a974-ab789a31f745 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.589541 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7ed1a-94f9-41e1-a974-ab789a31f745 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.595673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 135/276] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:25.599155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced5663-4024-4a25-b438-2dd2fef94fd7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.599460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced5663-4024-4a25-b438-2dd2fef94fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.599741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced5663-4024-4a25-b438-2dd2fef94fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.621877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1ced5663-4024-4a25-b438-2dd2fef94fd7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:25.623353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1ced5663-4024-4a25-b438-2dd2fef94fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:25.630124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aed7bb-0d32-4a7f-b690-be223026f01a None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:25.657585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:25.669826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708c443-22ce-40b9-a8dc-8e788b641881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:25.670689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708c443-22ce-40b9-a8dc-8e788b641881 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.673006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708c443-22ce-40b9-a8dc-8e788b641881 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.673006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708c443-22ce-40b9-a8dc-8e788b641881 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.684302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708c443-22ce-40b9-a8dc-8e788b641881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-e708c443-22ce-40b9-a8dc-8e788b641881 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:25.692595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:25.693558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:25.694561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 142/277] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:25.701330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708c443-22ce-40b9-a8dc-8e788b641881 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:25.711365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 136/278] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:25.734330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced5663-4024-4a25-b438-2dd2fef94fd7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:25.808322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:25.820139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708c443-22ce-40b9-a8dc-8e788b641881 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:25.862457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced5663-4024-4a25-b438-2dd2fef94fd7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:25.870148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2c8d4-8f7d-414c-bbc3-7c8417a4c43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:25.870910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2c8d4-8f7d-414c-bbc3-7c8417a4c43c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.871260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2c8d4-8f7d-414c-bbc3-7c8417a4c43c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.871604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2c8d4-8f7d-414c-bbc3-7c8417a4c43c None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.872429 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2c8d4-8f7d-414c-bbc3-7c8417a4c43c None admin] RBAC: Adding query filter params (name=member) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:25.872860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2c8d4-8f7d-414c-bbc3-7c8417a4c43c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:25.874309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2c8d4-8f7d-414c-bbc3-7c8417a4c43c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:25.885670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 143/279] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:25.891307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2c8d4-8f7d-414c-bbc3-7c8417a4c43c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:25.909139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275583f4-d062-4736-b007-a4e67dbb989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:25.910183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275583f4-d062-4736-b007-a4e67dbb989c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:25.911267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275583f4-d062-4736-b007-a4e67dbb989c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:25.911865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275583f4-d062-4736-b007-a4e67dbb989c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:25.924242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275583f4-d062-4736-b007-a4e67dbb989c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-275583f4-d062-4736-b007-a4e67dbb989c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:25.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:25.934441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:25.935257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:25.937865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 137/280] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:25.943482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:25.945988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275583f4-d062-4736-b007-a4e67dbb989c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:25.999348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2c8d4-8f7d-414c-bbc3-7c8417a4c43c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:26.042253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5ad20-cd51-450a-839d-07887e3b2ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:26.043128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5ad20-cd51-450a-839d-07887e3b2ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:26.043833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5ad20-cd51-450a-839d-07887e3b2ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:26.044121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5ad20-cd51-450a-839d-07887e3b2ab0 None admin] PATH_INFO: `/v3/users/f9e405d8f66a4216ab930122a293fb23` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:26.056285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275583f4-d062-4736-b007-a4e67dbb989c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:26.061411 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5ad20-cd51-450a-839d-07887e3b2ab0 None admin] RBAC: Authorizing `identity:get_user(user_id=f9e405d8f66a4216ab930122a293fb23)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:26.062872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a5ad20-cd51-450a-839d-07887e3b2ab0 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:26.065025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 144/281] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:25 2022] GET /identity/v3/users/f9e405d8f66a4216ab930122a293fb23 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:26.066453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5227d7-6611-407b-9934-d9ca136f7c7c None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:26.066739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5227d7-6611-407b-9934-d9ca136f7c7c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:26.067054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5227d7-6611-407b-9934-d9ca136f7c7c None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:26.069250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 145/282] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:26.070965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5227d7-6611-407b-9934-d9ca136f7c7c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:26.077156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:26.097514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f8ba3-d0cf-4d56-9f2c-b3c03b9b633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:26.099235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f8ba3-d0cf-4d56-9f2c-b3c03b9b633c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:26.099710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f8ba3-d0cf-4d56-9f2c-b3c03b9b633c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:26.100189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f8ba3-d0cf-4d56-9f2c-b3c03b9b633c None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:26.101326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f8ba3-d0cf-4d56-9f2c-b3c03b9b633c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:26.102072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f8ba3-d0cf-4d56-9f2c-b3c03b9b633c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:26.103767 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f8ba3-d0cf-4d56-9f2c-b3c03b9b633c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:26.113665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 138/283] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:26.115402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a9648-1454-4ebb-b6e7-c942f9a5de7d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:26.115883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a9648-1454-4ebb-b6e7-c942f9a5de7d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:26.116350 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a9648-1454-4ebb-b6e7-c942f9a5de7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:26.133855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a35a9648-1454-4ebb-b6e7-c942f9a5de7d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:26.135411 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a35a9648-1454-4ebb-b6e7-c942f9a5de7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:26.191514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5227d7-6611-407b-9934-d9ca136f7c7c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:26.196147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:26.222502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 139/284] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:26.225070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a9648-1454-4ebb-b6e7-c942f9a5de7d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:26.246137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95670b-dae5-407c-9017-51c0499fafa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:26.247270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95670b-dae5-407c-9017-51c0499fafa4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:26.247694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95670b-dae5-407c-9017-51c0499fafa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:26.248112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95670b-dae5-407c-9017-51c0499fafa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:26.249078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95670b-dae5-407c-9017-51c0499fafa4 None admin] RBAC: Adding query filter params (role.id=61e62f29eafd45c8b9fb99205a60bc4e, user.id=f9e405d8f66a4216ab930122a293fb23, scope.system=all) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:26.249561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95670b-dae5-407c-9017-51c0499fafa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:26.251265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95670b-dae5-407c-9017-51c0499fafa4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:26.257598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 146/285] 10.222.0.45 () {56 vars in 1308 bytes} [Tue Jul 5 15:50:26 2022] GET /identity/v3/role_assignments?user.id=f9e405d8f66a4216ab930122a293fb23&scope.system=all&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:26.258625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95670b-dae5-407c-9017-51c0499fafa4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:26.362614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a9648-1454-4ebb-b6e7-c942f9a5de7d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:26.372511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:26.407416 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d7410-bdc1-43e6-a617-5dcc397b3522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:26.408309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d7410-bdc1-43e6-a617-5dcc397b3522 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:26.408593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d7410-bdc1-43e6-a617-5dcc397b3522 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:26.409428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d7410-bdc1-43e6-a617-5dcc397b3522 None admin] PATH_INFO: `/v3/users/ea81b012b9064e21b61c0813b08f4e11` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:26.411085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d7410-bdc1-43e6-a617-5dcc397b3522 None admin] RBAC: Authorizing `identity:get_user(user_id=ea81b012b9064e21b61c0813b08f4e11)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:26.412952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d7410-bdc1-43e6-a617-5dcc397b3522 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:26.415247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 140/286] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:26 2022] GET /identity/v3/users/ea81b012b9064e21b61c0813b08f4e11 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:26.422644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d7410-bdc1-43e6-a617-5dcc397b3522 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:26.459909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95670b-dae5-407c-9017-51c0499fafa4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:26.532098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c774a71-2fcb-4f6d-b512-8604f06dc5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:26.532826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c774a71-2fcb-4f6d-b512-8604f06dc5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:26.533114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c774a71-2fcb-4f6d-b512-8604f06dc5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:26.533395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c774a71-2fcb-4f6d-b512-8604f06dc5a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:26.564961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c774a71-2fcb-4f6d-b512-8604f06dc5a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-5c774a71-2fcb-4f6d-b512-8604f06dc5a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:26.573347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:26.574248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:26.575017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 147/287] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:26.604445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c774a71-2fcb-4f6d-b512-8604f06dc5a3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:26.607577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d7410-bdc1-43e6-a617-5dcc397b3522 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:26.643519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4ef3f-cada-4a28-bb0b-ca58d19add24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:26.644756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4ef3f-cada-4a28-bb0b-ca58d19add24 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:26.645836 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4ef3f-cada-4a28-bb0b-ca58d19add24 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:26.646333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4ef3f-cada-4a28-bb0b-ca58d19add24 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:26.647740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4ef3f-cada-4a28-bb0b-ca58d19add24 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:26.649425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4ef3f-cada-4a28-bb0b-ca58d19add24 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:26.651341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 141/288] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:26 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:26.657365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4ef3f-cada-4a28-bb0b-ca58d19add24 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:26.667183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:26.709471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:26.767583 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4ef3f-cada-4a28-bb0b-ca58d19add24 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:26.802335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe037d51-681d-47b0-96b1-e880706d7825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:26.803545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe037d51-681d-47b0-96b1-e880706d7825 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:26.804044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe037d51-681d-47b0-96b1-e880706d7825 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:26.804514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe037d51-681d-47b0-96b1-e880706d7825 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/users/ea81b012b9064e21b61c0813b08f4e11/roles/583fe587d8d94b0fad411aeefb4181e0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:26.806745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe037d51-681d-47b0-96b1-e880706d7825 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a83413aa348d7a1e9fcd9b258330b, user_id=ea81b012b9064e21b61c0813b08f4e11, role_id=583fe587d8d94b0fad411aeefb4181e0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:26.808418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe037d51-681d-47b0-96b1-e880706d7825 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:26.821996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c774a71-2fcb-4f6d-b512-8604f06dc5a3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:26.866685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3a838-189f-47d8-bed8-d7f66414edba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:26.868194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3a838-189f-47d8-bed8-d7f66414edba None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:26.868435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3a838-189f-47d8-bed8-d7f66414edba None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:26.868823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3a838-189f-47d8-bed8-d7f66414edba None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:26.869698 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3a838-189f-47d8-bed8-d7f66414edba None admin] RBAC: Adding query filter params (name=member) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:26.870106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3a838-189f-47d8-bed8-d7f66414edba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:26.871645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3a838-189f-47d8-bed8-d7f66414edba None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:26.884489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 148/289] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:26.889840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3a838-189f-47d8-bed8-d7f66414edba None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:26.912422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 142/290] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:50:26 2022] PUT /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/users/ea81b012b9064e21b61c0813b08f4e11/roles/583fe587d8d94b0fad411aeefb4181e0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:27.013137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:27.029158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3a838-189f-47d8-bed8-d7f66414edba None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:27.077417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e17aae-3a47-4561-8d1a-ff9b8e0e207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:27.078654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e17aae-3a47-4561-8d1a-ff9b8e0e207b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:27.078938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e17aae-3a47-4561-8d1a-ff9b8e0e207b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:27.079221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e17aae-3a47-4561-8d1a-ff9b8e0e207b None admin] PATH_INFO: `/v3/users/7ae4a2043a0c463b8188ed634893d10c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:27.081561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e17aae-3a47-4561-8d1a-ff9b8e0e207b None admin] RBAC: Authorizing `identity:get_user(user_id=7ae4a2043a0c463b8188ed634893d10c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:27.083281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e17aae-3a47-4561-8d1a-ff9b8e0e207b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:27.085532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 149/291] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:26 2022] GET /identity/v3/users/7ae4a2043a0c463b8188ed634893d10c => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:27.091436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe037d51-681d-47b0-96b1-e880706d7825 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:27.210029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe037d51-681d-47b0-96b1-e880706d7825 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:27.213686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:27.247542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12eee46-eafb-48fb-8ba3-bc355abd9472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:27.248518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12eee46-eafb-48fb-8ba3-bc355abd9472 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:27.249034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12eee46-eafb-48fb-8ba3-bc355abd9472 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:27.249504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12eee46-eafb-48fb-8ba3-bc355abd9472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:27.318530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12eee46-eafb-48fb-8ba3-bc355abd9472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:27.320068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12eee46-eafb-48fb-8ba3-bc355abd9472 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:27.420353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 143/292] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:27.425781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e17aae-3a47-4561-8d1a-ff9b8e0e207b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:27.561373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e17aae-3a47-4561-8d1a-ff9b8e0e207b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:27.610865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb99cb5-fe19-481c-8c5c-6990c134f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:27.611738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb99cb5-fe19-481c-8c5c-6990c134f727 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:27.612027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb99cb5-fe19-481c-8c5c-6990c134f727 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:27.612303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb99cb5-fe19-481c-8c5c-6990c134f727 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:27.613538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb99cb5-fe19-481c-8c5c-6990c134f727 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:27.615005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb99cb5-fe19-481c-8c5c-6990c134f727 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:27.617045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 150/293] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:27 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:27.622531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12eee46-eafb-48fb-8ba3-bc355abd9472 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:27.624704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:27.772405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12eee46-eafb-48fb-8ba3-bc355abd9472 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:27.807875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d86e19-20ba-4371-8631-a9d4ecf871da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:27.808602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d86e19-20ba-4371-8631-a9d4ecf871da None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:27.808924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d86e19-20ba-4371-8631-a9d4ecf871da None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:27.809212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d86e19-20ba-4371-8631-a9d4ecf871da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:27.815697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d86e19-20ba-4371-8631-a9d4ecf871da None admin] RBAC: Adding query filter params (role.id=61e62f29eafd45c8b9fb99205a60bc4e, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b, user.id=7ae4a2043a0c463b8188ed634893d10c) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:27.815697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d86e19-20ba-4371-8631-a9d4ecf871da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:27.815697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d86e19-20ba-4371-8631-a9d4ecf871da None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:27.818061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 144/294] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:27 2022] GET /identity/v3/role_assignments?user.id=7ae4a2043a0c463b8188ed634893d10c&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:28.142273 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e863911-8c67-4db5-9e20-5477d0be8c21 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.142770 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e863911-8c67-4db5-9e20-5477d0be8c21 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.142770 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e863911-8c67-4db5-9e20-5477d0be8c21 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:28.144288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 151/295] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:28.150349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da590c-93f7-4452-b165-bbf36feaaea8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.151157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da590c-93f7-4452-b165-bbf36feaaea8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.151670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da590c-93f7-4452-b165-bbf36feaaea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:28.170900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-53da590c-93f7-4452-b165-bbf36feaaea8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:28.172414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-53da590c-93f7-4452-b165-bbf36feaaea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:28.239038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:28.252840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 145/296] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:28.269638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e863911-8c67-4db5-9e20-5477d0be8c21 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:28.305233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:28.345034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bf88f-c042-4d62-9ca0-ff24ae2bc58e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.345316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bf88f-c042-4d62-9ca0-ff24ae2bc58e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.346093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bf88f-c042-4d62-9ca0-ff24ae2bc58e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:28.347894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 146/297] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:50:28.354339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7383b-9abd-4c7b-8784-9b461ac62346 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.354843 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7383b-9abd-4c7b-8784-9b461ac62346 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.355293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7383b-9abd-4c7b-8784-9b461ac62346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:28.373015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c1f7383b-9abd-4c7b-8784-9b461ac62346 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:28.374493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c1f7383b-9abd-4c7b-8784-9b461ac62346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:28.398581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e863911-8c67-4db5-9e20-5477d0be8c21 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:28.439387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df869dc-5481-4f44-8b81-0b560c741480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:28.440281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df869dc-5481-4f44-8b81-0b560c741480 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.440566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df869dc-5481-4f44-8b81-0b560c741480 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.440871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df869dc-5481-4f44-8b81-0b560c741480 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:28.442163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df869dc-5481-4f44-8b81-0b560c741480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:28.443597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df869dc-5481-4f44-8b81-0b560c741480 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:28.449612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 152/298] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:28.449736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 147/299] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:28.458216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df869dc-5481-4f44-8b81-0b560c741480 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:28.478021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7383b-9abd-4c7b-8784-9b461ac62346 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:28.569796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:28.577623 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df869dc-5481-4f44-8b81-0b560c741480 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:28.597548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7383b-9abd-4c7b-8784-9b461ac62346 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:28.617784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b10972-d278-4831-84e3-774b9c3e19a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:28.618712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b10972-d278-4831-84e3-774b9c3e19a2 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.619142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b10972-d278-4831-84e3-774b9c3e19a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.619556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b10972-d278-4831-84e3-774b9c3e19a2 None admin] PATH_INFO: `/v3/groups` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:28.620503 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b10972-d278-4831-84e3-774b9c3e19a2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:28.622182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b10972-d278-4831-84e3-774b9c3e19a2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:28.626179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:28.635064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20176e-bf51-45f3-885f-8df937e48ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:28.635855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20176e-bf51-45f3-885f-8df937e48ffc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.636138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20176e-bf51-45f3-885f-8df937e48ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.636412 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20176e-bf51-45f3-885f-8df937e48ffc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:28.648187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20176e-bf51-45f3-885f-8df937e48ffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ee20176e-bf51-45f3-885f-8df937e48ffc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:28.656256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:28.657464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:28.658492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:28.660823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 148/300] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:28.666418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20176e-bf51-45f3-885f-8df937e48ffc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:28.695397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 153/301] 10.222.0.45 () {60 vars in 1134 bytes} [Tue Jul 5 15:50:28 2022] POST /identity/v3/groups => generated 229 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:28.695397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ec2fd-992f-4d7c-b972-07c034b902f7 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.695397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ec2fd-992f-4d7c-b972-07c034b902f7 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.695397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ec2fd-992f-4d7c-b972-07c034b902f7 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:28.695397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 154/302] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:28.700712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53b13a-6bec-44f1-a9c6-60488cdf769d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.701350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53b13a-6bec-44f1-a9c6-60488cdf769d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.701350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53b13a-6bec-44f1-a9c6-60488cdf769d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:28.723670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-2d53b13a-6bec-44f1-a9c6-60488cdf769d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:28.726958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2d53b13a-6bec-44f1-a9c6-60488cdf769d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:28.792320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20176e-bf51-45f3-885f-8df937e48ffc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:28.800209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 155/303] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:28.823784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53b13a-6bec-44f1-a9c6-60488cdf769d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:28.836924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431c152-536c-4652-97d0-965e4bb12285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:28.838075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c152-536c-4652-97d0-965e4bb12285 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.838536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c152-536c-4652-97d0-965e4bb12285 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.838985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c152-536c-4652-97d0-965e4bb12285 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:28.839890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431c152-536c-4652-97d0-965e4bb12285 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:28.840335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431c152-536c-4652-97d0-965e4bb12285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:28.842128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431c152-536c-4652-97d0-965e4bb12285 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:28.852267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 149/304] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:28.860518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431c152-536c-4652-97d0-965e4bb12285 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:28.913944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:28.922225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:28.941677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53b13a-6bec-44f1-a9c6-60488cdf769d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:28.982020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431c152-536c-4652-97d0-965e4bb12285 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:28.990393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b472c3-b272-4a2b-9bc1-0d0531163789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:28.991196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b472c3-b272-4a2b-9bc1-0d0531163789 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:28.991477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b472c3-b272-4a2b-9bc1-0d0531163789 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:28.991748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b472c3-b272-4a2b-9bc1-0d0531163789 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:29.007069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b472c3-b272-4a2b-9bc1-0d0531163789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-85b472c3-b272-4a2b-9bc1-0d0531163789 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:29.017871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:29.018747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:29.019591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 156/305] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:29.020790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d24694-c83a-4a73-b6ec-97636808a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:29.021670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24694-c83a-4a73-b6ec-97636808a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:29.021958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24694-c83a-4a73-b6ec-97636808a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:29.022234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d24694-c83a-4a73-b6ec-97636808a3b8 None admin] PATH_INFO: `/v3/users/484e304f0183405a934acad728340600` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:29.028467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b472c3-b272-4a2b-9bc1-0d0531163789 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:29.042402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d24694-c83a-4a73-b6ec-97636808a3b8 None admin] RBAC: Authorizing `identity:get_user(user_id=484e304f0183405a934acad728340600)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:29.044012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d24694-c83a-4a73-b6ec-97636808a3b8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:29.045941 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 150/306] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:28 2022] GET /identity/v3/users/484e304f0183405a934acad728340600 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:29.052177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d24694-c83a-4a73-b6ec-97636808a3b8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:29.147212 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b472c3-b272-4a2b-9bc1-0d0531163789 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:29.167992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d24694-c83a-4a73-b6ec-97636808a3b8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:29.177019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:29.190564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fe85d-5f42-4980-9356-ca215a711890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:29.192381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe85d-5f42-4980-9356-ca215a711890 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:29.192798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe85d-5f42-4980-9356-ca215a711890 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:29.193196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe85d-5f42-4980-9356-ca215a711890 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:29.194324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fe85d-5f42-4980-9356-ca215a711890 None admin] RBAC: Adding query filter params (name=member) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:29.194726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fe85d-5f42-4980-9356-ca215a711890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:29.196744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fe85d-5f42-4980-9356-ca215a711890 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:29.206803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819257af-d5b5-4ca7-b86e-cde9cb1031a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:29.208811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 157/307] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:29.212228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819257af-d5b5-4ca7-b86e-cde9cb1031a7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:29.212524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819257af-d5b5-4ca7-b86e-cde9cb1031a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:29.212838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819257af-d5b5-4ca7-b86e-cde9cb1031a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:29.213424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fe85d-5f42-4980-9356-ca215a711890 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:29.214738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819257af-d5b5-4ca7-b86e-cde9cb1031a7 None admin] RBAC: Adding query filter params (role.id=583fe587d8d94b0fad411aeefb4181e0, user.id=484e304f0183405a934acad728340600, scope.system=all) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:29.215082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819257af-d5b5-4ca7-b86e-cde9cb1031a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:29.216548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819257af-d5b5-4ca7-b86e-cde9cb1031a7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:29.222489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 151/308] 10.222.0.45 () {56 vars in 1308 bytes} [Tue Jul 5 15:50:29 2022] GET /identity/v3/role_assignments?user.id=484e304f0183405a934acad728340600&scope.system=all&role.id=583fe587d8d94b0fad411aeefb4181e0 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:29.224991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:29.338041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fe85d-5f42-4980-9356-ca215a711890 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:29.376959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32aa1c-5a93-461a-9dd3-697d64416754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:29.378287 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32aa1c-5a93-461a-9dd3-697d64416754 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:29.378762 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32aa1c-5a93-461a-9dd3-697d64416754 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:29.379228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32aa1c-5a93-461a-9dd3-697d64416754 None admin] PATH_INFO: `/v3/users/f9e405d8f66a4216ab930122a293fb23` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:29.380780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32aa1c-5a93-461a-9dd3-697d64416754 None admin] RBAC: Authorizing `identity:get_user(user_id=f9e405d8f66a4216ab930122a293fb23)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:29.382415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32aa1c-5a93-461a-9dd3-697d64416754 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:29.384607 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 158/309] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:29 2022] GET /identity/v3/users/f9e405d8f66a4216ab930122a293fb23 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:29.389839 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819257af-d5b5-4ca7-b86e-cde9cb1031a7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:29.495241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819257af-d5b5-4ca7-b86e-cde9cb1031a7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:29.529408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846688-c990-4286-b3b1-bd147d375e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:29.530270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846688-c990-4286-b3b1-bd147d375e14 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:29.530552 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846688-c990-4286-b3b1-bd147d375e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:29.530829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846688-c990-4286-b3b1-bd147d375e14 None admin] PATH_INFO: `/v3/system/users/f9e405d8f66a4216ab930122a293fb23/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:29.532338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51846688-c990-4286-b3b1-bd147d375e14 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f9e405d8f66a4216ab930122a293fb23, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:29.533764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51846688-c990-4286-b3b1-bd147d375e14 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:29.560298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748092a6-ffc8-4a3a-b976-eba803875c2c None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:29.560639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748092a6-ffc8-4a3a-b976-eba803875c2c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:29.560897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748092a6-ffc8-4a3a-b976-eba803875c2c None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:29.562433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 159/310] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:50:29.567286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7c345-bd67-476c-9382-a5db2b55b5bf None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:29.567570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7c345-bd67-476c-9382-a5db2b55b5bf None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:29.567846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7c345-bd67-476c-9382-a5db2b55b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:29.584538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d6e7c345-bd67-476c-9382-a5db2b55b5bf None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:29.585862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d6e7c345-bd67-476c-9382-a5db2b55b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:29.631977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 152/311] 10.222.0.45 () {58 vars in 1334 bytes} [Tue Jul 5 15:50:29 2022] PUT /identity/v3/system/users/f9e405d8f66a4216ab930122a293fb23/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:50:29.676959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 160/312] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:29.694078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846688-c990-4286-b3b1-bd147d375e14 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:29.732021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:29.813933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846688-c990-4286-b3b1-bd147d375e14 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:29.850650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df22a2-3ec2-45e6-8f2a-aec970fe967d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:29.851740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df22a2-3ec2-45e6-8f2a-aec970fe967d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:29.852088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df22a2-3ec2-45e6-8f2a-aec970fe967d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:29.852370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df22a2-3ec2-45e6-8f2a-aec970fe967d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:29.862978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df22a2-3ec2-45e6-8f2a-aec970fe967d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-32df22a2-3ec2-45e6-8f2a-aec970fe967d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:29.870847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:29.871729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:29.872508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 153/313] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:29.879083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7c345-bd67-476c-9382-a5db2b55b5bf None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:29.882380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:29.989361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7c345-bd67-476c-9382-a5db2b55b5bf None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:30.036237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e29ba7-a7eb-4bc7-9a73-7fd1659d5a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:30.036946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e29ba7-a7eb-4bc7-9a73-7fd1659d5a83 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:30.037235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e29ba7-a7eb-4bc7-9a73-7fd1659d5a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:30.037513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e29ba7-a7eb-4bc7-9a73-7fd1659d5a83 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:30.038265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e29ba7-a7eb-4bc7-9a73-7fd1659d5a83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:30.038572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e29ba7-a7eb-4bc7-9a73-7fd1659d5a83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:30.039945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e29ba7-a7eb-4bc7-9a73-7fd1659d5a83 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:30.048801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 161/314] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:30.054292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df22a2-3ec2-45e6-8f2a-aec970fe967d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:30.154344 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:30.160735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df22a2-3ec2-45e6-8f2a-aec970fe967d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:30.195031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5897e8-956e-442f-9f92-ac2ed7de9e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:30.195910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5897e8-956e-442f-9f92-ac2ed7de9e21 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:30.196192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5897e8-956e-442f-9f92-ac2ed7de9e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:30.196465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5897e8-956e-442f-9f92-ac2ed7de9e21 None admin] PATH_INFO: `/v3/users/ea81b012b9064e21b61c0813b08f4e11` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:30.197691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5897e8-956e-442f-9f92-ac2ed7de9e21 None admin] RBAC: Authorizing `identity:get_user(user_id=ea81b012b9064e21b61c0813b08f4e11)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:30.199152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5897e8-956e-442f-9f92-ac2ed7de9e21 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:30.200781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 154/315] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:30 2022] GET /identity/v3/users/ea81b012b9064e21b61c0813b08f4e11 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:30.210059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e29ba7-a7eb-4bc7-9a73-7fd1659d5a83 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:30.279043 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:30.328459 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e29ba7-a7eb-4bc7-9a73-7fd1659d5a83 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:30.371712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0841ed-000b-4520-8072-4f62dcf844a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:30.372366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0841ed-000b-4520-8072-4f62dcf844a0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:30.372647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0841ed-000b-4520-8072-4f62dcf844a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:30.372957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0841ed-000b-4520-8072-4f62dcf844a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:30.444622 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0841ed-000b-4520-8072-4f62dcf844a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:30.446099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0841ed-000b-4520-8072-4f62dcf844a0 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:30.512301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:30.546552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 162/316] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:30.552911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5897e8-956e-442f-9f92-ac2ed7de9e21 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:30.662335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5897e8-956e-442f-9f92-ac2ed7de9e21 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:30.697984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166d34f-0b48-42b8-b7fb-4c088b76f15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:30.698776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166d34f-0b48-42b8-b7fb-4c088b76f15b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:30.699062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166d34f-0b48-42b8-b7fb-4c088b76f15b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:30.699338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166d34f-0b48-42b8-b7fb-4c088b76f15b None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:30.700522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166d34f-0b48-42b8-b7fb-4c088b76f15b None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:30.702006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166d34f-0b48-42b8-b7fb-4c088b76f15b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:30.703992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 155/317] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:30 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:30.709625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0841ed-000b-4520-8072-4f62dcf844a0 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:30.810991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0841ed-000b-4520-8072-4f62dcf844a0 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:30.849200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce355273-c7ef-448b-ab19-d8268cdfdf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:30.849853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce355273-c7ef-448b-ab19-d8268cdfdf79 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:30.850136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce355273-c7ef-448b-ab19-d8268cdfdf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:30.850410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce355273-c7ef-448b-ab19-d8268cdfdf79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:30.851301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce355273-c7ef-448b-ab19-d8268cdfdf79 None admin] RBAC: Adding query filter params (role.id=583fe587d8d94b0fad411aeefb4181e0, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b, user.id=ea81b012b9064e21b61c0813b08f4e11) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:30.852109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce355273-c7ef-448b-ab19-d8268cdfdf79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:30.853654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce355273-c7ef-448b-ab19-d8268cdfdf79 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:30.864886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 163/318] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:30 2022] GET /identity/v3/role_assignments?user.id=ea81b012b9064e21b61c0813b08f4e11&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=583fe587d8d94b0fad411aeefb4181e0 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:31.003042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6dae9-4cf1-4ffd-b88b-7a3b82f572ad None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.003335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6dae9-4cf1-4ffd-b88b-7a3b82f572ad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.003508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6dae9-4cf1-4ffd-b88b-7a3b82f572ad None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.004969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 156/319] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:31.010339 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f1e30-ff49-4f2e-882c-695424adb00c None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.010634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f1e30-ff49-4f2e-882c-695424adb00c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.010913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f1e30-ff49-4f2e-882c-695424adb00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.028911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-0d9f1e30-ff49-4f2e-882c-695424adb00c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:31.030085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0d9f1e30-ff49-4f2e-882c-695424adb00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:31.057805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:31.100998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 164/320] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:31.124001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6dae9-4cf1-4ffd-b88b-7a3b82f572ad None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:31.155089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:31.228403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6dae9-4cf1-4ffd-b88b-7a3b82f572ad None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:31.262226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c605dd7-a3c5-4027-a985-9613ee37596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:31.262933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c605dd7-a3c5-4027-a985-9613ee37596d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.263210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c605dd7-a3c5-4027-a985-9613ee37596d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.263478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c605dd7-a3c5-4027-a985-9613ee37596d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.264631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c605dd7-a3c5-4027-a985-9613ee37596d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:31.266104 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c605dd7-a3c5-4027-a985-9613ee37596d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:31.267862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 157/321] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:31.273844 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f1e30-ff49-4f2e-882c-695424adb00c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:31.361788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac33a7-42ae-4fd9-9e12-8501ed88abc6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.362125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac33a7-42ae-4fd9-9e12-8501ed88abc6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.362340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac33a7-42ae-4fd9-9e12-8501ed88abc6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.364033 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 158/322] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:31.368755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c124778-de36-4a88-a5ad-c534ea4bdfcf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.369073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c124778-de36-4a88-a5ad-c534ea4bdfcf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.369346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c124778-de36-4a88-a5ad-c534ea4bdfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.383508 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f1e30-ff49-4f2e-882c-695424adb00c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:31.386135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5c124778-de36-4a88-a5ad-c534ea4bdfcf None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:31.387296 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5c124778-de36-4a88-a5ad-c534ea4bdfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:31.419605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2dc8d-90fb-4e4f-b55b-b4bf185bc2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:31.420271 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2dc8d-90fb-4e4f-b55b-b4bf185bc2b7 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.420555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2dc8d-90fb-4e4f-b55b-b4bf185bc2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.420855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2dc8d-90fb-4e4f-b55b-b4bf185bc2b7 None admin] PATH_INFO: `/v3/groups` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.421548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2dc8d-90fb-4e4f-b55b-b4bf185bc2b7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:31.423098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2dc8d-90fb-4e4f-b55b-b4bf185bc2b7 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:31.459005 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 159/323] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:31.481601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c124778-de36-4a88-a5ad-c534ea4bdfcf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:31.507728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:31.552338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 165/324] 10.222.0.45 () {60 vars in 1134 bytes} [Tue Jul 5 15:50:31 2022] POST /identity/v3/groups => generated 226 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:31.580967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c124778-de36-4a88-a5ad-c534ea4bdfcf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:31.619828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596305d-ca44-41d0-9807-f973daf1f515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:31.620635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596305d-ca44-41d0-9807-f973daf1f515 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.620949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596305d-ca44-41d0-9807-f973daf1f515 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.621222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596305d-ca44-41d0-9807-f973daf1f515 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.633433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596305d-ca44-41d0-9807-f973daf1f515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5596305d-ca44-41d0-9807-f973daf1f515 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:31.641137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:31.642074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:31.648789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 160/325] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:31.650814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2dc8d-90fb-4e4f-b55b-b4bf185bc2b7 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:31.675406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:31.761153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2dc8d-90fb-4e4f-b55b-b4bf185bc2b7 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:31.766959 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b70b5f-07c1-4683-9516-40491b4b4e8c None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.767247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b70b5f-07c1-4683-9516-40491b4b4e8c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.767527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b70b5f-07c1-4683-9516-40491b4b4e8c None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.768928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 161/326] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:50:31.774815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d596dda-8a26-46f6-9c7c-386d6edb550b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.775266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d596dda-8a26-46f6-9c7c-386d6edb550b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.775689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d596dda-8a26-46f6-9c7c-386d6edb550b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.795655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2d596dda-8a26-46f6-9c7c-386d6edb550b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:31.797039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2d596dda-8a26-46f6-9c7c-386d6edb550b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:31.803342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53656ae-9c0f-408d-996e-b122b405e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:31.803998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53656ae-9c0f-408d-996e-b122b405e359 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.804246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53656ae-9c0f-408d-996e-b122b405e359 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.804525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53656ae-9c0f-408d-996e-b122b405e359 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.805329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53656ae-9c0f-408d-996e-b122b405e359 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:31.805638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53656ae-9c0f-408d-996e-b122b405e359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:31.807529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53656ae-9c0f-408d-996e-b122b405e359 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:31.818697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 166/327] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:31.824929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53656ae-9c0f-408d-996e-b122b405e359 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:31.896075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 162/328] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:31.919382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d596dda-8a26-46f6-9c7c-386d6edb550b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:31.921593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:31.945052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:31.948030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53656ae-9c0f-408d-996e-b122b405e359 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:31.996045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12481f-fcff-4a1a-b27d-cd1efd26408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:31.997285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12481f-fcff-4a1a-b27d-cd1efd26408f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:31.997567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12481f-fcff-4a1a-b27d-cd1efd26408f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:31.997840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12481f-fcff-4a1a-b27d-cd1efd26408f None admin] PATH_INFO: `/v3/users/484e304f0183405a934acad728340600` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:31.999793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12481f-fcff-4a1a-b27d-cd1efd26408f None admin] RBAC: Authorizing `identity:get_user(user_id=484e304f0183405a934acad728340600)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:32.001416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12481f-fcff-4a1a-b27d-cd1efd26408f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:32.003392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 167/329] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:31 2022] GET /identity/v3/users/484e304f0183405a934acad728340600 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:32.010505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12481f-fcff-4a1a-b27d-cd1efd26408f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:32.059798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d596dda-8a26-46f6-9c7c-386d6edb550b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:32.099586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ba19a-5fa4-4ae7-99c7-f262f6054d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:32.100444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ba19a-5fa4-4ae7-99c7-f262f6054d43 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:32.100731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ba19a-5fa4-4ae7-99c7-f262f6054d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:32.101052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ba19a-5fa4-4ae7-99c7-f262f6054d43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:32.116672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ba19a-5fa4-4ae7-99c7-f262f6054d43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-118ba19a-5fa4-4ae7-99c7-f262f6054d43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:32.126301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:32.127212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:32.128203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 163/330] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:32.132702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12481f-fcff-4a1a-b27d-cd1efd26408f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:32.141779 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ba19a-5fa4-4ae7-99c7-f262f6054d43 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:32.174048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81873d-5fb4-40e8-b816-8fce3280c027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:32.174915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81873d-5fb4-40e8-b816-8fce3280c027 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:32.175200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81873d-5fb4-40e8-b816-8fce3280c027 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:32.175482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81873d-5fb4-40e8-b816-8fce3280c027 None admin] PATH_INFO: `/v3/system/users/484e304f0183405a934acad728340600/roles/583fe587d8d94b0fad411aeefb4181e0` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:32.178292 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81873d-5fb4-40e8-b816-8fce3280c027 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=484e304f0183405a934acad728340600, role_id=583fe587d8d94b0fad411aeefb4181e0)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:32.179885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81873d-5fb4-40e8-b816-8fce3280c027 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:32.247876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:32.252703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 168/331] 10.222.0.45 () {58 vars in 1334 bytes} [Tue Jul 5 15:50:32 2022] PUT /identity/v3/system/users/484e304f0183405a934acad728340600/roles/583fe587d8d94b0fad411aeefb4181e0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:32.275579 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ba19a-5fa4-4ae7-99c7-f262f6054d43 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:32.315697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107decb-3801-4d44-a65b-7d276a8f7769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:32.316326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107decb-3801-4d44-a65b-7d276a8f7769 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:32.316625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107decb-3801-4d44-a65b-7d276a8f7769 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:32.317479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107decb-3801-4d44-a65b-7d276a8f7769 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:32.318298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107decb-3801-4d44-a65b-7d276a8f7769 None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:32.318660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107decb-3801-4d44-a65b-7d276a8f7769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:32.320180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107decb-3801-4d44-a65b-7d276a8f7769 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:32.330366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 164/332] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:32.336386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81873d-5fb4-40e8-b816-8fce3280c027 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:32.356433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:32.455110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81873d-5fb4-40e8-b816-8fce3280c027 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:32.492444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb04a9-9d69-4940-bec0-21d0507f8795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:32.493732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb04a9-9d69-4940-bec0-21d0507f8795 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:32.494218 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb04a9-9d69-4940-bec0-21d0507f8795 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:32.494692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb04a9-9d69-4940-bec0-21d0507f8795 None admin] PATH_INFO: `/v3/users/f9e405d8f66a4216ab930122a293fb23` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:32.496081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb04a9-9d69-4940-bec0-21d0507f8795 None admin] RBAC: Authorizing `identity:get_user(user_id=f9e405d8f66a4216ab930122a293fb23)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:32.497794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb04a9-9d69-4940-bec0-21d0507f8795 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:32.500006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 169/333] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:32 2022] GET /identity/v3/users/f9e405d8f66a4216ab930122a293fb23 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:32.509895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107decb-3801-4d44-a65b-7d276a8f7769 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:32.653885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107decb-3801-4d44-a65b-7d276a8f7769 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:32.698394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4331230-a36f-44e2-9f43-556bbd42c66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:32.699054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4331230-a36f-44e2-9f43-556bbd42c66f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:32.699337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4331230-a36f-44e2-9f43-556bbd42c66f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:32.699605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4331230-a36f-44e2-9f43-556bbd42c66f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:32.700435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4331230-a36f-44e2-9f43-556bbd42c66f None admin] RBAC: Adding query filter params (role.id=61e62f29eafd45c8b9fb99205a60bc4e, user.id=f9e405d8f66a4216ab930122a293fb23, scope.system=all) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:32.700744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4331230-a36f-44e2-9f43-556bbd42c66f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:32.702217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4331230-a36f-44e2-9f43-556bbd42c66f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:32.720309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 165/334] 10.222.0.45 () {56 vars in 1308 bytes} [Tue Jul 5 15:50:32 2022] GET /identity/v3/role_assignments?user.id=f9e405d8f66a4216ab930122a293fb23&scope.system=all&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:32.722501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:34.082905 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab351faa-89e2-4661-8740-3700ac7b1d1b None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:34.083758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab351faa-89e2-4661-8740-3700ac7b1d1b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:34.084215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab351faa-89e2-4661-8740-3700ac7b1d1b None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:34.086039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 170/335] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:34.091481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed84639-85a1-465c-b6d4-5ec6da4b099e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:34.091895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed84639-85a1-465c-b6d4-5ec6da4b099e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:34.092039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed84639-85a1-465c-b6d4-5ec6da4b099e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:34.113216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0ed84639-85a1-465c-b6d4-5ec6da4b099e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:34.114984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0ed84639-85a1-465c-b6d4-5ec6da4b099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:34.208581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 166/336] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:34.254294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab351faa-89e2-4661-8740-3700ac7b1d1b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:34.367949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:34.388520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab351faa-89e2-4661-8740-3700ac7b1d1b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:34.424081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2ac5a-004a-4907-810a-ee58564ae9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:34.424899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2ac5a-004a-4907-810a-ee58564ae9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:34.425305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2ac5a-004a-4907-810a-ee58564ae9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:34.425582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2ac5a-004a-4907-810a-ee58564ae9f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:34.436489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2ac5a-004a-4907-810a-ee58564ae9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-5dc2ac5a-004a-4907-810a-ee58564ae9f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:34.444491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:34.445452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:34.446263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:34.450395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 171/337] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:34.451975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed84639-85a1-465c-b6d4-5ec6da4b099e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:34.526289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716082e-6b3d-4054-a7e7-a7e570c118ec None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:34.526942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716082e-6b3d-4054-a7e7-a7e570c118ec None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:34.527431 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716082e-6b3d-4054-a7e7-a7e570c118ec None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:34.529278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 172/338] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:34.536162 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2990d-37f3-468a-b4fb-4c4843fd8133 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:34.536726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2990d-37f3-468a-b4fb-4c4843fd8133 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:34.537239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2990d-37f3-468a-b4fb-4c4843fd8133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:34.586163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed84639-85a1-465c-b6d4-5ec6da4b099e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:34.588988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-23f2990d-37f3-468a-b4fb-4c4843fd8133 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:34.590423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-23f2990d-37f3-468a-b4fb-4c4843fd8133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:34.620857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:34.628661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567c849-8287-4f43-8f43-78842e46d57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:34.634293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567c849-8287-4f43-8f43-78842e46d57b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:34.634293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567c849-8287-4f43-8f43-78842e46d57b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:34.634293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567c849-8287-4f43-8f43-78842e46d57b None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:34.634293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567c849-8287-4f43-8f43-78842e46d57b None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:34.634293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567c849-8287-4f43-8f43-78842e46d57b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:34.634293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567c849-8287-4f43-8f43-78842e46d57b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:34.647970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 167/339] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:34.655262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567c849-8287-4f43-8f43-78842e46d57b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:34.801871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567c849-8287-4f43-8f43-78842e46d57b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:34.848834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697b974-7cec-4e68-82b2-e352170c83af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:34.849497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697b974-7cec-4e68-82b2-e352170c83af None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:34.849777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697b974-7cec-4e68-82b2-e352170c83af None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:34.850050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697b974-7cec-4e68-82b2-e352170c83af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:34.858348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 173/340] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:34.883907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2990d-37f3-468a-b4fb-4c4843fd8133 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:35.076992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:35.096800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697b974-7cec-4e68-82b2-e352170c83af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:35.096800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697b974-7cec-4e68-82b2-e352170c83af None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:35.157380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:35.234893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2990d-37f3-468a-b4fb-4c4843fd8133 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:35.260325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 168/341] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:35.268357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980937c6-30c4-4fad-b28f-6cb223c2bbc7 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:35.272600 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980937c6-30c4-4fad-b28f-6cb223c2bbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:35.274295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980937c6-30c4-4fad-b28f-6cb223c2bbc7 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:35.276613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 169/342] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:35 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:35.277500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980937c6-30c4-4fad-b28f-6cb223c2bbc7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:35.313396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087779a-5761-4f79-8428-4da7422bac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:35.314329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087779a-5761-4f79-8428-4da7422bac79 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:35.314602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087779a-5761-4f79-8428-4da7422bac79 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:35.314882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087779a-5761-4f79-8428-4da7422bac79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:35.328617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087779a-5761-4f79-8428-4da7422bac79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-9087779a-5761-4f79-8428-4da7422bac79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:35.337779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:35.342189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 174/343] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 456 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:35.343208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120629f-8923-4a19-997a-3575e29a813f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:35.343935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120629f-8923-4a19-997a-3575e29a813f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:35.343935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120629f-8923-4a19-997a-3575e29a813f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:35.360367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4120629f-8923-4a19-997a-3575e29a813f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:35.362677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4120629f-8923-4a19-997a-3575e29a813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:35.401549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:35.414192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980937c6-30c4-4fad-b28f-6cb223c2bbc7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:35.444975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 175/344] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:35.446533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf15c7-47ad-475a-b3f4-68d639671e1c None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:35.452785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf15c7-47ad-475a-b3f4-68d639671e1c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:35.453300 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf15c7-47ad-475a-b3f4-68d639671e1c None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:35.455114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 176/345] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:35.455983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf15c7-47ad-475a-b3f4-68d639671e1c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:35.459144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d3c00-1eac-41ab-b2e2-cbbe007958c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:35.460250 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d3c00-1eac-41ab-b2e2-cbbe007958c4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:35.460684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d3c00-1eac-41ab-b2e2-cbbe007958c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:35.461139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d3c00-1eac-41ab-b2e2-cbbe007958c4 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:35.462525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d3c00-1eac-41ab-b2e2-cbbe007958c4 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:35.464292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d3c00-1eac-41ab-b2e2-cbbe007958c4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:35.467407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 170/346] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:35 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:35.469619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5186-b647-4e90-aaaf-f42d23fde8bf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:35.470077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5186-b647-4e90-aaaf-f42d23fde8bf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:35.470491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5186-b647-4e90-aaaf-f42d23fde8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:35.507192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4a5e5186-b647-4e90-aaaf-f42d23fde8bf None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:35.509201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4a5e5186-b647-4e90-aaaf-f42d23fde8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:35.553447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:35.591998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 171/347] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:35.596795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e5186-b647-4e90-aaaf-f42d23fde8bf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:35.609968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf15c7-47ad-475a-b3f4-68d639671e1c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:35.661073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182a92f-c046-48be-bece-041b89bd51e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:35.661984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182a92f-c046-48be-bece-041b89bd51e9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:35.662440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182a92f-c046-48be-bece-041b89bd51e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:35.662866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182a92f-c046-48be-bece-041b89bd51e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:35.663826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182a92f-c046-48be-bece-041b89bd51e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:35.664415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182a92f-c046-48be-bece-041b89bd51e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:35.666190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182a92f-c046-48be-bece-041b89bd51e9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:35.681665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 177/348] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:35.685276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182a92f-c046-48be-bece-041b89bd51e9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:35.709620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:35.716868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e5186-b647-4e90-aaaf-f42d23fde8bf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:35.759841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd91db-4aaa-42b7-9d66-fc8e4cff0da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:35.760711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd91db-4aaa-42b7-9d66-fc8e4cff0da6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:35.761025 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd91db-4aaa-42b7-9d66-fc8e4cff0da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:35.761309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd91db-4aaa-42b7-9d66-fc8e4cff0da6 None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:35.767669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd91db-4aaa-42b7-9d66-fc8e4cff0da6 None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:35.769154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd91db-4aaa-42b7-9d66-fc8e4cff0da6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:35.771672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 172/349] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:35 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:35.772519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd91db-4aaa-42b7-9d66-fc8e4cff0da6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:35.816174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182a92f-c046-48be-bece-041b89bd51e9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:35.849834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:35.864008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab45fb-895e-47f9-b588-763225397fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:35.865916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab45fb-895e-47f9-b588-763225397fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:35.866505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab45fb-895e-47f9-b588-763225397fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:35.866973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab45fb-895e-47f9-b588-763225397fe3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:35.880500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab45fb-895e-47f9-b588-763225397fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-5dab45fb-895e-47f9-b588-763225397fe3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:35.890238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:35.891150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:35.891928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:35.897251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 178/350] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:35.897251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab45fb-895e-47f9-b588-763225397fe3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:35.909347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd91db-4aaa-42b7-9d66-fc8e4cff0da6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:35.946689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b857024-9038-4124-b1b7-1fdcf90ac432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:35.947851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b857024-9038-4124-b1b7-1fdcf90ac432 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:35.948290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b857024-9038-4124-b1b7-1fdcf90ac432 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:35.948708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b857024-9038-4124-b1b7-1fdcf90ac432 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:35.961110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b857024-9038-4124-b1b7-1fdcf90ac432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5b857024-9038-4124-b1b7-1fdcf90ac432 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:35.969045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:35.970114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:50:35.970954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:35.973763 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 173/351] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:35.974743 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b857024-9038-4124-b1b7-1fdcf90ac432 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:35.990986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:36.018001 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab45fb-895e-47f9-b588-763225397fe3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:36.067613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2382e4-d7cb-4a44-9cf3-5e51543714e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:36.068610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2382e4-d7cb-4a44-9cf3-5e51543714e8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.069088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2382e4-d7cb-4a44-9cf3-5e51543714e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.069544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2382e4-d7cb-4a44-9cf3-5e51543714e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.099365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b857024-9038-4124-b1b7-1fdcf90ac432 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:36.144167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:36.149215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fbdca-4974-41af-a045-3f1b89623524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:36.150206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fbdca-4974-41af-a045-3f1b89623524 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.150499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fbdca-4974-41af-a045-3f1b89623524 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.151044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fbdca-4974-41af-a045-3f1b89623524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.151899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fbdca-4974-41af-a045-3f1b89623524 None admin] RBAC: Adding query filter params (group.id=803eb07f8d704ccfb6c6a179b7510dbd, role.id=61e62f29eafd45c8b9fb99205a60bc4e, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:36.152372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fbdca-4974-41af-a045-3f1b89623524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:36.154179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fbdca-4974-41af-a045-3f1b89623524 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:36.161051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 174/352] 10.222.0.45 () {56 vars in 1376 bytes} [Tue Jul 5 15:50:35 2022] GET /identity/v3/role_assignments?group.id=803eb07f8d704ccfb6c6a179b7510dbd&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:36.161051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fbdca-4974-41af-a045-3f1b89623524 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:36.162212 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2382e4-d7cb-4a44-9cf3-5e51543714e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:36.163921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2382e4-d7cb-4a44-9cf3-5e51543714e8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:36.250418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:36.290056 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fbdca-4974-41af-a045-3f1b89623524 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:36.319617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 179/353] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:36.321263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3bbf0-d00d-4643-a0a9-c94c5a331d9e None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.321756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3bbf0-d00d-4643-a0a9-c94c5a331d9e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.322204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3bbf0-d00d-4643-a0a9-c94c5a331d9e None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.323749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 180/354] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:36.324722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3bbf0-d00d-4643-a0a9-c94c5a331d9e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:36.347168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741bf26-924c-4002-9073-a56650df565c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:36.347168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741bf26-924c-4002-9073-a56650df565c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.347168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741bf26-924c-4002-9073-a56650df565c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.347168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741bf26-924c-4002-9073-a56650df565c None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.348364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741bf26-924c-4002-9073-a56650df565c None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:36.348618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741bf26-924c-4002-9073-a56650df565c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:36.350055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741bf26-924c-4002-9073-a56650df565c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:36.361583 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 175/355] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:36.363255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d052f-1106-48f4-8ab2-7af0b420ba49 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.363624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d052f-1106-48f4-8ab2-7af0b420ba49 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.363973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d052f-1106-48f4-8ab2-7af0b420ba49 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.365776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 176/356] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:36.366576 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d052f-1106-48f4-8ab2-7af0b420ba49 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:36.428412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:36.445472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3bbf0-d00d-4643-a0a9-c94c5a331d9e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:36.496149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9255a-4b99-4526-893f-2457d37f4550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:36.497563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9255a-4b99-4526-893f-2457d37f4550 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.497856 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9255a-4b99-4526-893f-2457d37f4550 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.498173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9255a-4b99-4526-893f-2457d37f4550 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.499108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9255a-4b99-4526-893f-2457d37f4550 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:36.499750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9255a-4b99-4526-893f-2457d37f4550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:36.502097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9255a-4b99-4526-893f-2457d37f4550 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:36.508508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d052f-1106-48f4-8ab2-7af0b420ba49 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:36.513567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 181/357] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:36.514914 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa0d4a-807e-4e60-8578-bfbfefcfc67c None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.515210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa0d4a-807e-4e60-8578-bfbfefcfc67c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.515492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa0d4a-807e-4e60-8578-bfbfefcfc67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.538096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-21aa0d4a-807e-4e60-8578-bfbfefcfc67c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:36.539483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-21aa0d4a-807e-4e60-8578-bfbfefcfc67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:36.551756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79db0-9ace-463a-9273-ecab91e920a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:36.552623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79db0-9ace-463a-9273-ecab91e920a9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.552936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79db0-9ace-463a-9273-ecab91e920a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.553217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79db0-9ace-463a-9273-ecab91e920a9 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.554407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b79db0-9ace-463a-9273-ecab91e920a9 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:36.555858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b79db0-9ace-463a-9273-ecab91e920a9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:36.558053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 177/358] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:36 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:36.558821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79db0-9ace-463a-9273-ecab91e920a9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:36.569393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:36.620040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 182/359] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:36.620040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704f931-af64-41d4-82e4-344f02bdacda None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.620040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704f931-af64-41d4-82e4-344f02bdacda None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.620040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704f931-af64-41d4-82e4-344f02bdacda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.649231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8704f931-af64-41d4-82e4-344f02bdacda None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:36.650687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8704f931-af64-41d4-82e4-344f02bdacda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:36.678443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79db0-9ace-463a-9273-ecab91e920a9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:36.720564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75e8b7-59a9-4a6a-b8dd-493fe3759c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:36.721363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e8b7-59a9-4a6a-b8dd-493fe3759c22 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.721740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e8b7-59a9-4a6a-b8dd-493fe3759c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.722021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e8b7-59a9-4a6a-b8dd-493fe3759c22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.746143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 183/360] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:36.746997 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704f931-af64-41d4-82e4-344f02bdacda None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:36.757138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:36.811190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75e8b7-59a9-4a6a-b8dd-493fe3759c22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:36.812666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75e8b7-59a9-4a6a-b8dd-493fe3759c22 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:36.824498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:36.874144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704f931-af64-41d4-82e4-344f02bdacda None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:36.918438 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391c342-c56a-4d2f-9411-e8f499ca49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:36.919467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c342-c56a-4d2f-9411-e8f499ca49fc None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:36.919751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c342-c56a-4d2f-9411-e8f499ca49fc None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:36.921419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c342-c56a-4d2f-9411-e8f499ca49fc None admin] PATH_INFO: `/v3/users/484e304f0183405a934acad728340600` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:36.922369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391c342-c56a-4d2f-9411-e8f499ca49fc None admin] RBAC: Authorizing `identity:get_user(user_id=484e304f0183405a934acad728340600)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:36.923559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391c342-c56a-4d2f-9411-e8f499ca49fc None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:36.925270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 184/361] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:50:36 2022] GET /identity/v3/users/484e304f0183405a934acad728340600 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:36.926605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391c342-c56a-4d2f-9411-e8f499ca49fc None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:36.956730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 178/362] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:36.957641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75e8b7-59a9-4a6a-b8dd-493fe3759c22 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:37.018131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:37.054908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391c342-c56a-4d2f-9411-e8f499ca49fc None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:37.086544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75e8b7-59a9-4a6a-b8dd-493fe3759c22 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:37.098160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c2d9a-9d8b-4ef6-9392-55cec4ac82ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:37.098773 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:37.099042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c2d9a-9d8b-4ef6-9392-55cec4ac82ca None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:37.099322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c2d9a-9d8b-4ef6-9392-55cec4ac82ca None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:37.099598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c2d9a-9d8b-4ef6-9392-55cec4ac82ca None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:37.102050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c2d9a-9d8b-4ef6-9392-55cec4ac82ca None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:37.103702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c2d9a-9d8b-4ef6-9392-55cec4ac82ca None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:37.105070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 185/363] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:36 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:37.105837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c2d9a-9d8b-4ef6-9392-55cec4ac82ca None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:37.134375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116456af-521f-45b0-bece-e3699880d59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:37.135192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116456af-521f-45b0-bece-e3699880d59a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:37.135477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116456af-521f-45b0-bece-e3699880d59a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:37.135753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116456af-521f-45b0-bece-e3699880d59a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:37.149290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116456af-521f-45b0-bece-e3699880d59a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-116456af-521f-45b0-bece-e3699880d59a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:37.159659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:37.161145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:37.161932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:37.162536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 179/364] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:37.163355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116456af-521f-45b0-bece-e3699880d59a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:37.243177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c2d9a-9d8b-4ef6-9392-55cec4ac82ca None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:37.283286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abfd8f-5c2f-4b35-9fa8-ea64e4dc2a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:37.284008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abfd8f-5c2f-4b35-9fa8-ea64e4dc2a09 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:37.284294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abfd8f-5c2f-4b35-9fa8-ea64e4dc2a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:37.284579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abfd8f-5c2f-4b35-9fa8-ea64e4dc2a09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:37.303732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abfd8f-5c2f-4b35-9fa8-ea64e4dc2a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:37.310975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116456af-521f-45b0-bece-e3699880d59a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-e1abfd8f-5c2f-4b35-9fa8-ea64e4dc2a09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:37.311568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:37.312418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:37.313557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 186/365] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:50:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:37.314210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abfd8f-5c2f-4b35-9fa8-ea64e4dc2a09 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:37.320037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:37.356077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d65f-0886-4e54-9f2c-fc350c731c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:37.356894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d65f-0886-4e54-9f2c-fc350c731c5a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:37.357186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d65f-0886-4e54-9f2c-fc350c731c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:37.357970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d65f-0886-4e54-9f2c-fc350c731c5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:37.358422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57d65f-0886-4e54-9f2c-fc350c731c5a None admin] RBAC: Adding query filter params (role.id=583fe587d8d94b0fad411aeefb4181e0, user.id=484e304f0183405a934acad728340600, scope.system=all) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:37.358744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57d65f-0886-4e54-9f2c-fc350c731c5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:37.360255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57d65f-0886-4e54-9f2c-fc350c731c5a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:37.377915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 180/366] 10.222.0.45 () {56 vars in 1308 bytes} [Tue Jul 5 15:50:37 2022] GET /identity/v3/role_assignments?user.id=484e304f0183405a934acad728340600&scope.system=all&role.id=583fe587d8d94b0fad411aeefb4181e0 => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:37.379143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d65f-0886-4e54-9f2c-fc350c731c5a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:37.418922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:37.446580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abfd8f-5c2f-4b35-9fa8-ea64e4dc2a09 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:37.489990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a7e22-9a81-4002-b2f7-021e2c7b6234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:37.490867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a7e22-9a81-4002-b2f7-021e2c7b6234 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:37.491156 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a7e22-9a81-4002-b2f7-021e2c7b6234 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:37.491431 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a7e22-9a81-4002-b2f7-021e2c7b6234 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:37.492985 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a7e22-9a81-4002-b2f7-021e2c7b6234 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:37.494422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a7e22-9a81-4002-b2f7-021e2c7b6234 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:37.497720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 187/367] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:37 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:37.498738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a7e22-9a81-4002-b2f7-021e2c7b6234 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:37.519037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d65f-0886-4e54-9f2c-fc350c731c5a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:37.581389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c74d3c-574f-4caa-8e2c-26fbebb8b1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:37.582499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c74d3c-574f-4caa-8e2c-26fbebb8b1ba None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:37.582800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c74d3c-574f-4caa-8e2c-26fbebb8b1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:37.583078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c74d3c-574f-4caa-8e2c-26fbebb8b1ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:37.585045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c74d3c-574f-4caa-8e2c-26fbebb8b1ba None admin] RBAC: Adding query filter params (group.id=803eb07f8d704ccfb6c6a179b7510dbd, role.id=2991af796dcd45e2b86208339339b977, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:37.585045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c74d3c-574f-4caa-8e2c-26fbebb8b1ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:37.588144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c74d3c-574f-4caa-8e2c-26fbebb8b1ba None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:37.595172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 181/368] 10.222.0.45 () {56 vars in 1376 bytes} [Tue Jul 5 15:50:37 2022] GET /identity/v3/role_assignments?group.id=803eb07f8d704ccfb6c6a179b7510dbd&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=2991af796dcd45e2b86208339339b977 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:37.595761 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c74d3c-574f-4caa-8e2c-26fbebb8b1ba None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:37.595967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:37.647449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a7e22-9a81-4002-b2f7-021e2c7b6234 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:37.693375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323dc6e-784f-48b4-a3ec-09cbef4dc457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:37.694107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323dc6e-784f-48b4-a3ec-09cbef4dc457 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:37.694597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323dc6e-784f-48b4-a3ec-09cbef4dc457 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:37.695008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323dc6e-784f-48b4-a3ec-09cbef4dc457 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:37.695781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323dc6e-784f-48b4-a3ec-09cbef4dc457 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:37.696120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323dc6e-784f-48b4-a3ec-09cbef4dc457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:37.698810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323dc6e-784f-48b4-a3ec-09cbef4dc457 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:37.711889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 188/369] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:37.712786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323dc6e-784f-48b4-a3ec-09cbef4dc457 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:37.722108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:37.745216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c74d3c-574f-4caa-8e2c-26fbebb8b1ba None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:37.802948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaf5f7-6135-4366-95f2-6d80c3c50d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:37.803628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaf5f7-6135-4366-95f2-6d80c3c50d4e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:37.803911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaf5f7-6135-4366-95f2-6d80c3c50d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:37.804189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaf5f7-6135-4366-95f2-6d80c3c50d4e None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:37.805698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aaf5f7-6135-4366-95f2-6d80c3c50d4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:37.806018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aaf5f7-6135-4366-95f2-6d80c3c50d4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:37.807947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aaf5f7-6135-4366-95f2-6d80c3c50d4e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:37.820427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 182/370] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:37.822734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaf5f7-6135-4366-95f2-6d80c3c50d4e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:37.852916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323dc6e-784f-48b4-a3ec-09cbef4dc457 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:37.884271 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:37.889726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39629d-0463-4ac8-8d49-da7106defd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:37.890647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39629d-0463-4ac8-8d49-da7106defd38 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:37.890928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39629d-0463-4ac8-8d49-da7106defd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:37.891202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39629d-0463-4ac8-8d49-da7106defd38 None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:37.892433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39629d-0463-4ac8-8d49-da7106defd38 None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:37.895682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39629d-0463-4ac8-8d49-da7106defd38 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:37.902334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 189/371] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:37 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:37.903301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39629d-0463-4ac8-8d49-da7106defd38 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:37.955148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaf5f7-6135-4366-95f2-6d80c3c50d4e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:38.008933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e928d-2ccf-4af9-bcf2-129fa1e46116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:38.009894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e928d-2ccf-4af9-bcf2-129fa1e46116 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:38.010354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e928d-2ccf-4af9-bcf2-129fa1e46116 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:38.010823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e928d-2ccf-4af9-bcf2-129fa1e46116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:38.024527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39629d-0463-4ac8-8d49-da7106defd38 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:38.032178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:38.062168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df8c72-7a10-44d9-9c83-e2c1906ba96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:38.062976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df8c72-7a10-44d9-9c83-e2c1906ba96f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:38.063405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df8c72-7a10-44d9-9c83-e2c1906ba96f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:38.063754 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df8c72-7a10-44d9-9c83-e2c1906ba96f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:38.089841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e928d-2ccf-4af9-bcf2-129fa1e46116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:38.091748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e928d-2ccf-4af9-bcf2-129fa1e46116 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:38.140428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df8c72-7a10-44d9-9c83-e2c1906ba96f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:38.141938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df8c72-7a10-44d9-9c83-e2c1906ba96f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:38.145071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:38.206502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 183/372] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:38.208051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e928d-2ccf-4af9-bcf2-129fa1e46116 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:38.258230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 190/373] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:38.264146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df8c72-7a10-44d9-9c83-e2c1906ba96f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:38.275744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:38.334720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e928d-2ccf-4af9-bcf2-129fa1e46116 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:38.372523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a22ad4-fc93-4450-a828-34f93babf5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:38.373567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22ad4-fc93-4450-a828-34f93babf5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:38.374054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22ad4-fc93-4450-a828-34f93babf5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:38.374523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22ad4-fc93-4450-a828-34f93babf5a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:38.375580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a22ad4-fc93-4450-a828-34f93babf5a3 None admin] RBAC: Adding query filter params (group.id=803eb07f8d704ccfb6c6a179b7510dbd, role.id=61e62f29eafd45c8b9fb99205a60bc4e, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:38.376096 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a22ad4-fc93-4450-a828-34f93babf5a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:38.377619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df8c72-7a10-44d9-9c83-e2c1906ba96f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:38.378034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a22ad4-fc93-4450-a828-34f93babf5a3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:38.386156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 184/374] 10.222.0.45 () {56 vars in 1376 bytes} [Tue Jul 5 15:50:38 2022] GET /identity/v3/role_assignments?group.id=803eb07f8d704ccfb6c6a179b7510dbd&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:38.386850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:38.389039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a22ad4-fc93-4450-a828-34f93babf5a3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:38.450954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8e557-05fe-40a7-bbef-41e4e6028662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:38.451839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8e557-05fe-40a7-bbef-41e4e6028662 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:38.452290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8e557-05fe-40a7-bbef-41e4e6028662 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:38.452736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8e557-05fe-40a7-bbef-41e4e6028662 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:38.454368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8e557-05fe-40a7-bbef-41e4e6028662 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:38.456143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8e557-05fe-40a7-bbef-41e4e6028662 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:38.463060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 191/375] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:38 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:38.465002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c128e-309d-48b3-825b-ff888f4c7a93 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:38.465544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c128e-309d-48b3-825b-ff888f4c7a93 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:38.466065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c128e-309d-48b3-825b-ff888f4c7a93 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:38.467779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 192/376] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:38.469330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c128e-309d-48b3-825b-ff888f4c7a93 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:38.529113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a22ad4-fc93-4450-a828-34f93babf5a3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:38.549210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:38.572319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8816a69-5407-47b9-9e1b-3f0e6884d858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:38.573495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8816a69-5407-47b9-9e1b-3f0e6884d858 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:38.573783 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8816a69-5407-47b9-9e1b-3f0e6884d858 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:38.574062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8816a69-5407-47b9-9e1b-3f0e6884d858 None admin] PATH_INFO: `/v3/projects/35018c3fbb4f440c9fd930631a71e6c6` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:38.575380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8816a69-5407-47b9-9e1b-3f0e6884d858 None admin] RBAC: Authorizing `identity:get_project(project_id=35018c3fbb4f440c9fd930631a71e6c6)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:38.576895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8816a69-5407-47b9-9e1b-3f0e6884d858 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:38.579481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 185/377] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:38 2022] GET /identity/v3/projects/35018c3fbb4f440c9fd930631a71e6c6 => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:38.580945 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f2833-03f1-4fe2-a757-7a08dc6cb811 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:38.581422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f2833-03f1-4fe2-a757-7a08dc6cb811 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:38.581875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f2833-03f1-4fe2-a757-7a08dc6cb811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:38.604433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e73f2833-03f1-4fe2-a757-7a08dc6cb811 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:38.605701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e73f2833-03f1-4fe2-a757-7a08dc6cb811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:38.614752 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c128e-309d-48b3-825b-ff888f4c7a93 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:38.654326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593296d1-e78f-40ff-bffe-7b6e108aec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:38.655661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593296d1-e78f-40ff-bffe-7b6e108aec8b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:38.656159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593296d1-e78f-40ff-bffe-7b6e108aec8b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:38.656635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593296d1-e78f-40ff-bffe-7b6e108aec8b None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:38.659351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593296d1-e78f-40ff-bffe-7b6e108aec8b None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:38.661333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593296d1-e78f-40ff-bffe-7b6e108aec8b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:38.663233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 193/378] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:38 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:38.664454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593296d1-e78f-40ff-bffe-7b6e108aec8b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:38.679122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 186/379] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:38.680085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f2833-03f1-4fe2-a757-7a08dc6cb811 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:38.727777 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:38.801981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f2833-03f1-4fe2-a757-7a08dc6cb811 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:38.816199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:38.820286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593296d1-e78f-40ff-bffe-7b6e108aec8b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:38.841054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e49c2-0d99-4371-8434-26b910344925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:38.841957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e49c2-0d99-4371-8434-26b910344925 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:38.842495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e49c2-0d99-4371-8434-26b910344925 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:38.842913 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e49c2-0d99-4371-8434-26b910344925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:38.844040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e49c2-0d99-4371-8434-26b910344925 None admin] RBAC: Adding query filter params (group.id=803eb07f8d704ccfb6c6a179b7510dbd, role.id=2991af796dcd45e2b86208339339b977, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:38.844498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e49c2-0d99-4371-8434-26b910344925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:38.846132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e49c2-0d99-4371-8434-26b910344925 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:38.853672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 187/380] 10.222.0.45 () {56 vars in 1376 bytes} [Tue Jul 5 15:50:38 2022] GET /identity/v3/role_assignments?group.id=803eb07f8d704ccfb6c6a179b7510dbd&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=2991af796dcd45e2b86208339339b977 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:38.854664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e49c2-0d99-4371-8434-26b910344925 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:38.869562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb0b2f-4b5a-4d4e-bc34-ddc11145f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:38.870726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb0b2f-4b5a-4d4e-bc34-ddc11145f154 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:38.871185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb0b2f-4b5a-4d4e-bc34-ddc11145f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:38.871632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb0b2f-4b5a-4d4e-bc34-ddc11145f154 None admin] PATH_INFO: `/v3/groups/d70ff95ce3524ded8aede53e07be5534` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:38.892386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb0b2f-4b5a-4d4e-bc34-ddc11145f154 None admin] RBAC: Authorizing `identity:get_group(group_id=d70ff95ce3524ded8aede53e07be5534)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:38.894333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb0b2f-4b5a-4d4e-bc34-ddc11145f154 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:38.899924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 194/381] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:38 2022] GET /identity/v3/groups/d70ff95ce3524ded8aede53e07be5534 => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:38.906144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb0b2f-4b5a-4d4e-bc34-ddc11145f154 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:39.067421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e49c2-0d99-4371-8434-26b910344925 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:39.097550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:39.104463 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb0b2f-4b5a-4d4e-bc34-ddc11145f154 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:39.118408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ceadc-8b38-4a27-9fea-6c265b048dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:39.119603 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ceadc-8b38-4a27-9fea-6c265b048dce None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:39.120099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ceadc-8b38-4a27-9fea-6c265b048dce None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:39.120475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ceadc-8b38-4a27-9fea-6c265b048dce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:39.133995 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ceadc-8b38-4a27-9fea-6c265b048dce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:39.146947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a44b5-26ac-4584-863f-5df5062c885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:39.149338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a44b5-26ac-4584-863f-5df5062c885d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:39.149338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a44b5-26ac-4584-863f-5df5062c885d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:39.149338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a44b5-26ac-4584-863f-5df5062c885d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:39.149812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a44b5-26ac-4584-863f-5df5062c885d None admin] RBAC: Adding query filter params (group.id=d70ff95ce3524ded8aede53e07be5534, role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=35018c3fbb4f440c9fd930631a71e6c6) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:39.150360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a44b5-26ac-4584-863f-5df5062c885d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5a1ceadc-8b38-4a27-9fea-6c265b048dce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:39.150727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:39.151558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:39.152337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 188/382] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:39.157694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a44b5-26ac-4584-863f-5df5062c885d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:39.165448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ceadc-8b38-4a27-9fea-6c265b048dce None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:39.168170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 195/383] 10.222.0.45 () {56 vars in 1376 bytes} [Tue Jul 5 15:50:38 2022] GET /identity/v3/role_assignments?group.id=d70ff95ce3524ded8aede53e07be5534&scope.project.id=35018c3fbb4f440c9fd930631a71e6c6&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 260 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:39.228660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:39.344214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ceadc-8b38-4a27-9fea-6c265b048dce None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:39.392587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56cc25-0c48-4f09-a57d-74cf17ccb92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:39.393305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56cc25-0c48-4f09-a57d-74cf17ccb92f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:39.393590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56cc25-0c48-4f09-a57d-74cf17ccb92f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:39.393871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56cc25-0c48-4f09-a57d-74cf17ccb92f None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:39.395203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56cc25-0c48-4f09-a57d-74cf17ccb92f None admin] RBAC: Adding query filter params (name=member) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:39.395546 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56cc25-0c48-4f09-a57d-74cf17ccb92f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:39.396934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56cc25-0c48-4f09-a57d-74cf17ccb92f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:39.408881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 189/384] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:39.414419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a44b5-26ac-4584-863f-5df5062c885d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:39.556051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a44b5-26ac-4584-863f-5df5062c885d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:39.592620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b8bf4-72a2-4a62-b275-766b7d4d1590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:39.593515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b8bf4-72a2-4a62-b275-766b7d4d1590 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:39.593806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b8bf4-72a2-4a62-b275-766b7d4d1590 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:39.594086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b8bf4-72a2-4a62-b275-766b7d4d1590 None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:39.595325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b8bf4-72a2-4a62-b275-766b7d4d1590 None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:39.596720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b8bf4-72a2-4a62-b275-766b7d4d1590 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:39.598742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 196/385] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:39 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:39.604304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56cc25-0c48-4f09-a57d-74cf17ccb92f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:39.740871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:39.751427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56cc25-0c48-4f09-a57d-74cf17ccb92f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:39.798857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b204509-5663-47d4-b7a5-3572189c996d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:39.799734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b204509-5663-47d4-b7a5-3572189c996d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:39.800021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b204509-5663-47d4-b7a5-3572189c996d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:39.800298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b204509-5663-47d4-b7a5-3572189c996d None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:39.802190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b204509-5663-47d4-b7a5-3572189c996d None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:39.803995 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b204509-5663-47d4-b7a5-3572189c996d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:39.807459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 190/386] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:39 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:39.813063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b8bf4-72a2-4a62-b275-766b7d4d1590 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:39.871812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:39.932307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c439cc2-9926-418c-b902-f7d406481414 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:39.932594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c439cc2-9926-418c-b902-f7d406481414 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:39.932925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c439cc2-9926-418c-b902-f7d406481414 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:39.934548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 191/387] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:39.939524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4299f-ab01-4291-a65a-0198f528f2b1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:39.939815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4299f-ab01-4291-a65a-0198f528f2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:39.940092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4299f-ab01-4291-a65a-0198f528f2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:39.954784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b8bf4-72a2-4a62-b275-766b7d4d1590 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:39.970024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-afa4299f-ab01-4291-a65a-0198f528f2b1 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:39.971223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-afa4299f-ab01-4291-a65a-0198f528f2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:39.991071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f330d9-3af4-4409-b2b9-1283c7a3f475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:39.991982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f330d9-3af4-4409-b2b9-1283c7a3f475 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:39.992265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f330d9-3af4-4409-b2b9-1283c7a3f475 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:39.992546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f330d9-3af4-4409-b2b9-1283c7a3f475 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/groups/803eb07f8d704ccfb6c6a179b7510dbd/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:39.994548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f330d9-3af4-4409-b2b9-1283c7a3f475 None admin] RBAC: Authorizing `identity:create_grant(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, group_id=803eb07f8d704ccfb6c6a179b7510dbd, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:39.996051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f330d9-3af4-4409-b2b9-1283c7a3f475 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:40.058157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 192/388] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:40.080415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4299f-ab01-4291-a65a-0198f528f2b1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:40.110825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 197/389] 10.222.0.45 () {58 vars in 1442 bytes} [Tue Jul 5 15:50:39 2022] PUT /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/groups/803eb07f8d704ccfb6c6a179b7510dbd/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:40.213645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:40.220156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4299f-ab01-4291-a65a-0198f528f2b1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:40.276260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd3372-0808-4701-99a7-901145c56286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:40.277945 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3372-0808-4701-99a7-901145c56286 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:40.278383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3372-0808-4701-99a7-901145c56286 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:40.278832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3372-0808-4701-99a7-901145c56286 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:40.305502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd3372-0808-4701-99a7-901145c56286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2dbd3372-0808-4701-99a7-901145c56286 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:40.313718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:40.314619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:40.315463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:40.318187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 193/390] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:40.320985 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f330d9-3af4-4409-b2b9-1283c7a3f475 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:40.429882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:40.441055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f330d9-3af4-4409-b2b9-1283c7a3f475 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:40.476077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c3c88-ca34-4b4f-8a56-3d469ca43419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:40.476991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3c88-ca34-4b4f-8a56-3d469ca43419 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:40.477275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3c88-ca34-4b4f-8a56-3d469ca43419 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:40.477549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c3c88-ca34-4b4f-8a56-3d469ca43419 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:40.478291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c3c88-ca34-4b4f-8a56-3d469ca43419 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:40.479860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c3c88-ca34-4b4f-8a56-3d469ca43419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:40.481300 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c3c88-ca34-4b4f-8a56-3d469ca43419 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:40.493042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 198/391] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:40.495372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd3372-0808-4701-99a7-901145c56286 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:40.630382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd3372-0808-4701-99a7-901145c56286 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:40.669449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff072d-423a-4973-b72c-3f087f776fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:40.670309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff072d-423a-4973-b72c-3f087f776fab None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:40.670592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff072d-423a-4973-b72c-3f087f776fab None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:40.670865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff072d-423a-4973-b72c-3f087f776fab None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:40.672035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff072d-423a-4973-b72c-3f087f776fab None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:40.673613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff072d-423a-4973-b72c-3f087f776fab None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:40.680102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 194/392] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:40 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:40.681342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c3c88-ca34-4b4f-8a56-3d469ca43419 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:40.825248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c3c88-ca34-4b4f-8a56-3d469ca43419 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:40.855332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c3fd7-b966-4a7a-9e42-baeb6031351e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:40.856557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c3fd7-b966-4a7a-9e42-baeb6031351e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:40.857072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c3fd7-b966-4a7a-9e42-baeb6031351e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:40.857547 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c3fd7-b966-4a7a-9e42-baeb6031351e None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:40.858976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c3fd7-b966-4a7a-9e42-baeb6031351e None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:40.860621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c3fd7-b966-4a7a-9e42-baeb6031351e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:40.862984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 199/393] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:40 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:40.869015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff072d-423a-4973-b72c-3f087f776fab None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:40.896856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:41.007955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff072d-423a-4973-b72c-3f087f776fab None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:41.055294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12ad33-958a-4741-9ffc-3df3f0191ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:41.056196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12ad33-958a-4741-9ffc-3df3f0191ad9 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:41.056697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12ad33-958a-4741-9ffc-3df3f0191ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:41.056820 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12ad33-958a-4741-9ffc-3df3f0191ad9 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/groups/803eb07f8d704ccfb6c6a179b7510dbd/roles/2991af796dcd45e2b86208339339b977` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:41.064127 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12ad33-958a-4741-9ffc-3df3f0191ad9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5, group_id=803eb07f8d704ccfb6c6a179b7510dbd, role_id=2991af796dcd45e2b86208339339b977)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:41.064127 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12ad33-958a-4741-9ffc-3df3f0191ad9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:41.132050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 195/394] 10.222.0.45 () {58 vars in 1442 bytes} [Tue Jul 5 15:50:40 2022] PUT /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5/groups/803eb07f8d704ccfb6c6a179b7510dbd/roles/2991af796dcd45e2b86208339339b977 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:41.203458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760745f5-3457-418d-b532-41725f2ab7cf None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:41.203792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760745f5-3457-418d-b532-41725f2ab7cf None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:41.204028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760745f5-3457-418d-b532-41725f2ab7cf None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:41.205725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 200/395] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:41.211252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baedb8b-d296-4615-9fe2-a21cc4c39aa8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:41.211551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baedb8b-d296-4615-9fe2-a21cc4c39aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:41.211837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baedb8b-d296-4615-9fe2-a21cc4c39aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:41.230302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9baedb8b-d296-4615-9fe2-a21cc4c39aa8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:41.231508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9baedb8b-d296-4615-9fe2-a21cc4c39aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:41.333431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 196/396] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:41.370286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760745f5-3457-418d-b532-41725f2ab7cf None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:41.396613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:41.470932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e688407-a007-4c1e-952f-dcee1cc23f56 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:41.471248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e688407-a007-4c1e-952f-dcee1cc23f56 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:41.471505 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e688407-a007-4c1e-952f-dcee1cc23f56 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:41.473695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 197/397] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:41.480946 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350297cb-e5f2-48c0-aaaa-ce420b384fae None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:41.481238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350297cb-e5f2-48c0-aaaa-ce420b384fae None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:41.481518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350297cb-e5f2-48c0-aaaa-ce420b384fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:41.493897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760745f5-3457-418d-b532-41725f2ab7cf None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:41.502045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-350297cb-e5f2-48c0-aaaa-ce420b384fae None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:41.503825 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-350297cb-e5f2-48c0-aaaa-ce420b384fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:41.517122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:41.553757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16defb68-088a-4820-a3b9-e25140629357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:41.554556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16defb68-088a-4820-a3b9-e25140629357 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:41.554835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16defb68-088a-4820-a3b9-e25140629357 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:41.555198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16defb68-088a-4820-a3b9-e25140629357 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:41.579989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16defb68-088a-4820-a3b9-e25140629357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-16defb68-088a-4820-a3b9-e25140629357 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:41.598656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:41.599635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:41.600447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 201/398] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:41.609882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16defb68-088a-4820-a3b9-e25140629357 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:41.630582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 198/399] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:41.660971 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350297cb-e5f2-48c0-aaaa-ce420b384fae None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:41.760385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:41.796555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16defb68-088a-4820-a3b9-e25140629357 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:41.845961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350297cb-e5f2-48c0-aaaa-ce420b384fae None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:41.861332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d308a-c476-4835-a322-ccc694de5062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:41.862349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d308a-c476-4835-a322-ccc694de5062 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:41.862637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d308a-c476-4835-a322-ccc694de5062 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:41.862917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d308a-c476-4835-a322-ccc694de5062 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:41.863746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d308a-c476-4835-a322-ccc694de5062 None admin] RBAC: Adding query filter params (name=member) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:41.864059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d308a-c476-4835-a322-ccc694de5062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:41.869211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d308a-c476-4835-a322-ccc694de5062 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:41.875721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 202/400] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:41.877032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0b307-dc8c-483c-89e5-ec0c8a76300b None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:41.877321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0b307-dc8c-483c-89e5-ec0c8a76300b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:41.877597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0b307-dc8c-483c-89e5-ec0c8a76300b None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:41.878843 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 203/401] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:41.884608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0b307-dc8c-483c-89e5-ec0c8a76300b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:41.902161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8790d-8e6b-401e-accf-978d83acc252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:41.903321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8790d-8e6b-401e-accf-978d83acc252 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:41.903829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8790d-8e6b-401e-accf-978d83acc252 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:41.904307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8790d-8e6b-401e-accf-978d83acc252 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:41.915546 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8790d-8e6b-401e-accf-978d83acc252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7fb8790d-8e6b-401e-accf-978d83acc252 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:41.923834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:41.924705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:41.925838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 199/402] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:41.926770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb858ae-f9f6-4d54-a5b3-a5ca6499f4cd None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:41.931642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb858ae-f9f6-4d54-a5b3-a5ca6499f4cd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:41.931942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb858ae-f9f6-4d54-a5b3-a5ca6499f4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:41.945105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:41.949274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8cb858ae-f9f6-4d54-a5b3-a5ca6499f4cd None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:41.950486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8cb858ae-f9f6-4d54-a5b3-a5ca6499f4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:42.004578 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0b307-dc8c-483c-89e5-ec0c8a76300b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:42.021933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 200/403] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:42.022854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb858ae-f9f6-4d54-a5b3-a5ca6499f4cd None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:42.043163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6abf0b-2337-436b-bc33-f1cef5cb9b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:42.044373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6abf0b-2337-436b-bc33-f1cef5cb9b4b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.048248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6abf0b-2337-436b-bc33-f1cef5cb9b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.048945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6abf0b-2337-436b-bc33-f1cef5cb9b4b None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.050494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6abf0b-2337-436b-bc33-f1cef5cb9b4b None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:42.052286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6abf0b-2337-436b-bc33-f1cef5cb9b4b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:42.054493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 204/404] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:41 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:42.055627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6abf0b-2337-436b-bc33-f1cef5cb9b4b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:42.071505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:42.137648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb858ae-f9f6-4d54-a5b3-a5ca6499f4cd None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:42.167318 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6abf0b-2337-436b-bc33-f1cef5cb9b4b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:42.175455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141c2ff-baf8-4f3f-9872-bebbd346d06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:42.176500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141c2ff-baf8-4f3f-9872-bebbd346d06a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.176623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141c2ff-baf8-4f3f-9872-bebbd346d06a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.176853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141c2ff-baf8-4f3f-9872-bebbd346d06a None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.177611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141c2ff-baf8-4f3f-9872-bebbd346d06a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:42.177921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141c2ff-baf8-4f3f-9872-bebbd346d06a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:42.179407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141c2ff-baf8-4f3f-9872-bebbd346d06a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:42.180874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:42.189158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 201/405] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:42.189996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141c2ff-baf8-4f3f-9872-bebbd346d06a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:42.208114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52a9ef-10a6-4774-a803-c1415c5b4d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:42.209316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52a9ef-10a6-4774-a803-c1415c5b4d11 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.209804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52a9ef-10a6-4774-a803-c1415c5b4d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.210280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52a9ef-10a6-4774-a803-c1415c5b4d11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.221309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52a9ef-10a6-4774-a803-c1415c5b4d11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-2f52a9ef-10a6-4774-a803-c1415c5b4d11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:42.229863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:42.230716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:42.231500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:42.233965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 205/406] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:50:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:42.234984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52a9ef-10a6-4774-a803-c1415c5b4d11 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:42.302581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141c2ff-baf8-4f3f-9872-bebbd346d06a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:42.317837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:42.339196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613627c-fdb7-41c3-8d9a-5584ecb2c820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:42.340058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613627c-fdb7-41c3-8d9a-5584ecb2c820 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.340349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613627c-fdb7-41c3-8d9a-5584ecb2c820 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.340630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613627c-fdb7-41c3-8d9a-5584ecb2c820 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.341874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613627c-fdb7-41c3-8d9a-5584ecb2c820 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:42.343330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613627c-fdb7-41c3-8d9a-5584ecb2c820 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:42.345358 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 202/407] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:42 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:42.345930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613627c-fdb7-41c3-8d9a-5584ecb2c820 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:42.354333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52a9ef-10a6-4774-a803-c1415c5b4d11 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:42.391823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c0ed4-5803-4966-ae17-42ea65cbb157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:42.393085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c0ed4-5803-4966-ae17-42ea65cbb157 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.393605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c0ed4-5803-4966-ae17-42ea65cbb157 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.394150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c0ed4-5803-4966-ae17-42ea65cbb157 None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.396198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c0ed4-5803-4966-ae17-42ea65cbb157 None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:42.398082 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c0ed4-5803-4966-ae17-42ea65cbb157 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:42.400232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 206/408] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:42 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:42.401288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c0ed4-5803-4966-ae17-42ea65cbb157 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:42.423160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:42.467684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613627c-fdb7-41c3-8d9a-5584ecb2c820 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:42.505143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf68e1f-81ef-47fb-b9ad-a29c99a0e013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:42.505919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf68e1f-81ef-47fb-b9ad-a29c99a0e013 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.506324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf68e1f-81ef-47fb-b9ad-a29c99a0e013 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.506602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf68e1f-81ef-47fb-b9ad-a29c99a0e013 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.507465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf68e1f-81ef-47fb-b9ad-a29c99a0e013 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:42.507756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf68e1f-81ef-47fb-b9ad-a29c99a0e013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:42.509239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf68e1f-81ef-47fb-b9ad-a29c99a0e013 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:42.517158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c0ed4-5803-4966-ae17-42ea65cbb157 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:42.518194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 203/409] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:42.519008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf68e1f-81ef-47fb-b9ad-a29c99a0e013 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:42.555657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99101c15-4d13-4ac5-b94d-08d52580c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:42.557173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99101c15-4d13-4ac5-b94d-08d52580c845 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.557708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99101c15-4d13-4ac5-b94d-08d52580c845 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.558221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99101c15-4d13-4ac5-b94d-08d52580c845 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/groups/803eb07f8d704ccfb6c6a179b7510dbd/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.560461 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99101c15-4d13-4ac5-b94d-08d52580c845 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a83413aa348d7a1e9fcd9b258330b, group_id=803eb07f8d704ccfb6c6a179b7510dbd, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:42.562271 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99101c15-4d13-4ac5-b94d-08d52580c845 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:42.625140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf68e1f-81ef-47fb-b9ad-a29c99a0e013 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:42.651156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:42.659547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb600e8-c4f6-49b4-bbe3-e60969fe5b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:42.660446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb600e8-c4f6-49b4-bbe3-e60969fe5b0f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.660735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb600e8-c4f6-49b4-bbe3-e60969fe5b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.661047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb600e8-c4f6-49b4-bbe3-e60969fe5b0f None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.661461 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:42.662343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb600e8-c4f6-49b4-bbe3-e60969fe5b0f None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:42.663841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb600e8-c4f6-49b4-bbe3-e60969fe5b0f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:42.677280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 204/410] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:42 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:42.677410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 207/411] 10.222.0.45 () {58 vars in 1442 bytes} [Tue Jul 5 15:50:42 2022] PUT /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/groups/803eb07f8d704ccfb6c6a179b7510dbd/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:42.677486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb600e8-c4f6-49b4-bbe3-e60969fe5b0f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:42.677553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890514a-181b-4b26-9c27-eca260c7e791 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.677553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890514a-181b-4b26-9c27-eca260c7e791 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.677553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890514a-181b-4b26-9c27-eca260c7e791 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.677553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 208/412] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:42.677553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a8b0f-ec0a-47c2-b79e-37d5a5bff9ba None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.677553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a8b0f-ec0a-47c2-b79e-37d5a5bff9ba None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.677888 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a8b0f-ec0a-47c2-b79e-37d5a5bff9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.699111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-9e0a8b0f-ec0a-47c2-b79e-37d5a5bff9ba None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:42.700885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9e0a8b0f-ec0a-47c2-b79e-37d5a5bff9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:42.798252 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 209/413] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:42.799173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a8b0f-ec0a-47c2-b79e-37d5a5bff9ba None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:42.830304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb600e8-c4f6-49b4-bbe3-e60969fe5b0f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:42.875822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f84b04-71ad-4c93-bbdf-a8971bb9a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:42.877076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f84b04-71ad-4c93-bbdf-a8971bb9a93b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:42.877563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f84b04-71ad-4c93-bbdf-a8971bb9a93b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:42.878037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f84b04-71ad-4c93-bbdf-a8971bb9a93b None admin] PATH_INFO: `/v3/groups/d70ff95ce3524ded8aede53e07be5534` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:42.879461 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f84b04-71ad-4c93-bbdf-a8971bb9a93b None admin] RBAC: Authorizing `identity:get_group(group_id=d70ff95ce3524ded8aede53e07be5534)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:42.881231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f84b04-71ad-4c93-bbdf-a8971bb9a93b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:42.883374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 205/414] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:42 2022] GET /identity/v3/groups/d70ff95ce3524ded8aede53e07be5534 => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:42.884344 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f84b04-71ad-4c93-bbdf-a8971bb9a93b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:42.952015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a8b0f-ec0a-47c2-b79e-37d5a5bff9ba None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:42.953331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:42.992358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:42.996451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f84b04-71ad-4c93-bbdf-a8971bb9a93b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:43.000381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c4800-092b-4065-aa2e-492efcea9f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:43.001293 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c4800-092b-4065-aa2e-492efcea9f49 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:43.001581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c4800-092b-4065-aa2e-492efcea9f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:43.001855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c4800-092b-4065-aa2e-492efcea9f49 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/groups/803eb07f8d704ccfb6c6a179b7510dbd/roles/2991af796dcd45e2b86208339339b977` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:43.007073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c4800-092b-4065-aa2e-492efcea9f49 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca8a83413aa348d7a1e9fcd9b258330b, group_id=803eb07f8d704ccfb6c6a179b7510dbd, role_id=2991af796dcd45e2b86208339339b977)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:43.007073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c4800-092b-4065-aa2e-492efcea9f49 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:43.036548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620873-4a90-4e42-8fba-532bf3401e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:43.037736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620873-4a90-4e42-8fba-532bf3401e25 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:43.038249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620873-4a90-4e42-8fba-532bf3401e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:43.038767 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620873-4a90-4e42-8fba-532bf3401e25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:43.050552 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac620873-4a90-4e42-8fba-532bf3401e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ac620873-4a90-4e42-8fba-532bf3401e25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:43.059244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:43.060119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:43.060910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:43.063294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 206/415] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:43.064263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620873-4a90-4e42-8fba-532bf3401e25 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:43.081089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 210/416] 10.222.0.45 () {58 vars in 1442 bytes} [Tue Jul 5 15:50:42 2022] PUT /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b/groups/803eb07f8d704ccfb6c6a179b7510dbd/roles/2991af796dcd45e2b86208339339b977 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:50:43.082378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c4800-092b-4065-aa2e-492efcea9f49 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:43.208889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620873-4a90-4e42-8fba-532bf3401e25 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:43.217844 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c4800-092b-4065-aa2e-492efcea9f49 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:43.238312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:43.260806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8f4bf-99e3-44fb-8b16-cdde40fbafe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:43.261704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8f4bf-99e3-44fb-8b16-cdde40fbafe0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:43.261990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8f4bf-99e3-44fb-8b16-cdde40fbafe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:43.262265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8f4bf-99e3-44fb-8b16-cdde40fbafe0 None admin] PATH_INFO: `/v3/projects/35018c3fbb4f440c9fd930631a71e6c6` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:43.263696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8f4bf-99e3-44fb-8b16-cdde40fbafe0 None admin] RBAC: Authorizing `identity:get_project(project_id=35018c3fbb4f440c9fd930631a71e6c6)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:43.265195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8f4bf-99e3-44fb-8b16-cdde40fbafe0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:43.266931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 207/417] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:43 2022] GET /identity/v3/projects/35018c3fbb4f440c9fd930631a71e6c6 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:43.273252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8f4bf-99e3-44fb-8b16-cdde40fbafe0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:43.288595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7471a-5e9e-4d45-935a-233009a96894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:43.289305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7471a-5e9e-4d45-935a-233009a96894 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:43.289594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7471a-5e9e-4d45-935a-233009a96894 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:43.290356 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7471a-5e9e-4d45-935a-233009a96894 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:43.291153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7471a-5e9e-4d45-935a-233009a96894 None admin] RBAC: Adding query filter params (name=member) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:43.291468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7471a-5e9e-4d45-935a-233009a96894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:43.292886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7471a-5e9e-4d45-935a-233009a96894 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:43.304849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 211/418] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:43.310022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7471a-5e9e-4d45-935a-233009a96894 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:43.374222 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:43.416744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8f4bf-99e3-44fb-8b16-cdde40fbafe0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:43.455653 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a34021-e3bf-4f12-99c0-952c9c6d4b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:43.456671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a34021-e3bf-4f12-99c0-952c9c6d4b33 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:43.457521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a34021-e3bf-4f12-99c0-952c9c6d4b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:43.457798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a34021-e3bf-4f12-99c0-952c9c6d4b33 None admin] PATH_INFO: `/v3/projects/35018c3fbb4f440c9fd930631a71e6c6/groups/d70ff95ce3524ded8aede53e07be5534/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:43.459856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a34021-e3bf-4f12-99c0-952c9c6d4b33 None admin] RBAC: Authorizing `identity:create_grant(project_id=35018c3fbb4f440c9fd930631a71e6c6, group_id=d70ff95ce3524ded8aede53e07be5534, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:43.461440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a34021-e3bf-4f12-99c0-952c9c6d4b33 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:43.462620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7471a-5e9e-4d45-935a-233009a96894 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:43.509695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63d452-a774-491b-8fab-90b0090401b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:43.510346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63d452-a774-491b-8fab-90b0090401b2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:43.510629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63d452-a774-491b-8fab-90b0090401b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:43.510904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63d452-a774-491b-8fab-90b0090401b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:43.538174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 208/419] 10.222.0.45 () {58 vars in 1442 bytes} [Tue Jul 5 15:50:43 2022] PUT /identity/v3/projects/35018c3fbb4f440c9fd930631a71e6c6/groups/d70ff95ce3524ded8aede53e07be5534/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:50:43.602874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63d452-a774-491b-8fab-90b0090401b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:43.604352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63d452-a774-491b-8fab-90b0090401b2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:43.615733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1831fb-faeb-4073-a0c2-3e1ffd365672 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:43.616030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1831fb-faeb-4073-a0c2-3e1ffd365672 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:43.616281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1831fb-faeb-4073-a0c2-3e1ffd365672 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:43.618603 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 209/420] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:43.629605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98998abe-aaf5-4399-a156-d7346e4117e7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:43.630256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98998abe-aaf5-4399-a156-d7346e4117e7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:43.630637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98998abe-aaf5-4399-a156-d7346e4117e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:43.666263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-98998abe-aaf5-4399-a156-d7346e4117e7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:43.667489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-98998abe-aaf5-4399-a156-d7346e4117e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:43.716953 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:43.755728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:43.781962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 210/421] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:43.787554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 212/422] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:50:43.795459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98998abe-aaf5-4399-a156-d7346e4117e7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:43.809758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63d452-a774-491b-8fab-90b0090401b2 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:43.921144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98998abe-aaf5-4399-a156-d7346e4117e7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:43.978111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63d452-a774-491b-8fab-90b0090401b2 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:43.984413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18470352-f61f-455d-9808-53ab77e4473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:43.985698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18470352-f61f-455d-9808-53ab77e4473f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:43.991393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18470352-f61f-455d-9808-53ab77e4473f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:43.991891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18470352-f61f-455d-9808-53ab77e4473f None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:43.993474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18470352-f61f-455d-9808-53ab77e4473f None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:43.995112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18470352-f61f-455d-9808-53ab77e4473f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:43.997044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 211/423] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:43 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:44.005840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18470352-f61f-455d-9808-53ab77e4473f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:44.029245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c3fb-2727-47f9-a74d-ee19c9e03a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:44.030062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c3fb-2727-47f9-a74d-ee19c9e03a2c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:44.030352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c3fb-2727-47f9-a74d-ee19c9e03a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:44.030632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c3fb-2727-47f9-a74d-ee19c9e03a2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:44.041569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859c3fb-2727-47f9-a74d-ee19c9e03a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-3859c3fb-2727-47f9-a74d-ee19c9e03a2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:44.050015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:44.051456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:44.052263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 213/424] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:44.061798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c3fb-2727-47f9-a74d-ee19c9e03a2c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:44.091272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:44.098321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:44.182224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18470352-f61f-455d-9808-53ab77e4473f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:44.247833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a67fb7-d4ae-434c-aa73-8732f4202e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:44.249105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a67fb7-d4ae-434c-aa73-8732f4202e5f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:44.250139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a67fb7-d4ae-434c-aa73-8732f4202e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:44.250705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a67fb7-d4ae-434c-aa73-8732f4202e5f None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:44.254284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a67fb7-d4ae-434c-aa73-8732f4202e5f None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:44.256022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a67fb7-d4ae-434c-aa73-8732f4202e5f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:44.258321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 212/425] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:44 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:44.260663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c3fb-2727-47f9-a74d-ee19c9e03a2c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:44.268744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a67fb7-d4ae-434c-aa73-8732f4202e5f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:44.327666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe8daf-aae6-4411-9bf8-9ee88e85062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:44.328570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8daf-aae6-4411-9bf8-9ee88e85062f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:44.329138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8daf-aae6-4411-9bf8-9ee88e85062f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:44.330401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8daf-aae6-4411-9bf8-9ee88e85062f None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:44.331292 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe8daf-aae6-4411-9bf8-9ee88e85062f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:44.335923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe8daf-aae6-4411-9bf8-9ee88e85062f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:44.337478 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe8daf-aae6-4411-9bf8-9ee88e85062f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:44.349924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 214/426] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:44.356038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe8daf-aae6-4411-9bf8-9ee88e85062f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:44.450452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a67fb7-d4ae-434c-aa73-8732f4202e5f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:44.473532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:44.497601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade5a83-9ce2-454b-8fbf-54895f69874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:44.498725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade5a83-9ce2-454b-8fbf-54895f69874f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:44.499185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade5a83-9ce2-454b-8fbf-54895f69874f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:44.499706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade5a83-9ce2-454b-8fbf-54895f69874f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:44.501144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade5a83-9ce2-454b-8fbf-54895f69874f None admin] RBAC: Adding query filter params (group.id=803eb07f8d704ccfb6c6a179b7510dbd, role.id=61e62f29eafd45c8b9fb99205a60bc4e, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:44.501683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade5a83-9ce2-454b-8fbf-54895f69874f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:44.503197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade5a83-9ce2-454b-8fbf-54895f69874f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:44.509736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 213/427] 10.222.0.45 () {56 vars in 1376 bytes} [Tue Jul 5 15:50:44 2022] GET /identity/v3/role_assignments?group.id=803eb07f8d704ccfb6c6a179b7510dbd&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 611 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:44.512228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:44.542681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe8daf-aae6-4411-9bf8-9ee88e85062f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:44.581345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa819c1f-5b29-49d5-990c-8be0e416585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:44.582273 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa819c1f-5b29-49d5-990c-8be0e416585c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:44.582697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa819c1f-5b29-49d5-990c-8be0e416585c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:44.583110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa819c1f-5b29-49d5-990c-8be0e416585c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:44.657380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa819c1f-5b29-49d5-990c-8be0e416585c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:44.659511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa819c1f-5b29-49d5-990c-8be0e416585c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:44.756328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:44.788597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 215/428] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:44.794021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade5a83-9ce2-454b-8fbf-54895f69874f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:44.917563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade5a83-9ce2-454b-8fbf-54895f69874f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:44.955400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da379e9f-7920-45fb-9baa-94fea4756491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:44.956288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da379e9f-7920-45fb-9baa-94fea4756491 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:44.956573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da379e9f-7920-45fb-9baa-94fea4756491 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:44.957471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da379e9f-7920-45fb-9baa-94fea4756491 None admin] PATH_INFO: `/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:44.958776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da379e9f-7920-45fb-9baa-94fea4756491 None admin] RBAC: Authorizing `identity:get_project(project_id=a45a2c2e6a0a4f7bac8faa658cb8bfb5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:44.960268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da379e9f-7920-45fb-9baa-94fea4756491 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:44.962513 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 214/429] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:44 2022] GET /identity/v3/projects/a45a2c2e6a0a4f7bac8faa658cb8bfb5 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:44.968107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa819c1f-5b29-49d5-990c-8be0e416585c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:45.083815 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa819c1f-5b29-49d5-990c-8be0e416585c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:45.123060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9266b-9a77-445c-af6c-16a7ed263433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:45.123921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9266b-9a77-445c-af6c-16a7ed263433 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:45.124207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9266b-9a77-445c-af6c-16a7ed263433 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:45.124482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9266b-9a77-445c-af6c-16a7ed263433 None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:45.125709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9266b-9a77-445c-af6c-16a7ed263433 None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:45.127291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9266b-9a77-445c-af6c-16a7ed263433 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:45.129803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 216/430] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:44 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:45.135622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da379e9f-7920-45fb-9baa-94fea4756491 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:45.195149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:45.211357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd232-ea15-4592-98f7-06cae046daea None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:45.211636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd232-ea15-4592-98f7-06cae046daea None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:45.218139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd232-ea15-4592-98f7-06cae046daea None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:45.218139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 217/431] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:45.219823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0c135-92da-4db8-8851-f52dd5504276 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:45.220110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0c135-92da-4db8-8851-f52dd5504276 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:45.220384 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0c135-92da-4db8-8851-f52dd5504276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:45.241302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-64f0c135-92da-4db8-8851-f52dd5504276 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:45.242898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-64f0c135-92da-4db8-8851-f52dd5504276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:45.266293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da379e9f-7920-45fb-9baa-94fea4756491 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:45.294171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:45.320887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616da5f-697c-4f2f-8e6f-2f182a358faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:45.322390 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616da5f-697c-4f2f-8e6f-2f182a358faf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:45.323010 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616da5f-697c-4f2f-8e6f-2f182a358faf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:45.323600 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616da5f-697c-4f2f-8e6f-2f182a358faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:45.324806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616da5f-697c-4f2f-8e6f-2f182a358faf None admin] RBAC: Adding query filter params (group.id=803eb07f8d704ccfb6c6a179b7510dbd, role.id=2991af796dcd45e2b86208339339b977, scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:45.325619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616da5f-697c-4f2f-8e6f-2f182a358faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:45.327758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616da5f-697c-4f2f-8e6f-2f182a358faf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:45.330183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 218/432] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:45.335393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 215/433] 10.222.0.45 () {56 vars in 1376 bytes} [Tue Jul 5 15:50:45 2022] GET /identity/v3/role_assignments?group.id=803eb07f8d704ccfb6c6a179b7510dbd&scope.project.id=a45a2c2e6a0a4f7bac8faa658cb8bfb5&role.id=2991af796dcd45e2b86208339339b977 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:45.361660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0c135-92da-4db8-8851-f52dd5504276 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:45.478222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b441dd2-536e-42fa-b98a-1f80415c04b2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:45.478940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b441dd2-536e-42fa-b98a-1f80415c04b2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:45.478940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b441dd2-536e-42fa-b98a-1f80415c04b2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:45.479269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0c135-92da-4db8-8851-f52dd5504276 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:45.484809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 216/434] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:45.486750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02565-a016-4539-9b86-119aa657f8cb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:45.487146 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02565-a016-4539-9b86-119aa657f8cb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:45.487530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02565-a016-4539-9b86-119aa657f8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:45.507463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ebc02565-a016-4539-9b86-119aa657f8cb None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:45.508403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ebc02565-a016-4539-9b86-119aa657f8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:45.544418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f6b3d-bafe-4873-a903-164bb1aae1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:45.545634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f6b3d-bafe-4873-a903-164bb1aae1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:45.546240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f6b3d-bafe-4873-a903-164bb1aae1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:45.546715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f6b3d-bafe-4873-a903-164bb1aae1b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:45.559335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f6b3d-bafe-4873-a903-164bb1aae1b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-6c7f6b3d-bafe-4873-a903-164bb1aae1b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:45.568423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:45.569340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:50:45.570112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:45.574285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 219/435] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:45.578565 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f6b3d-bafe-4873-a903-164bb1aae1b8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:45.621301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 217/436] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:45.621887 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:45.648192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc02565-a016-4539-9b86-119aa657f8cb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:45.703896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f6b3d-bafe-4873-a903-164bb1aae1b8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:45.706899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:45.742879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1b559-7cd0-485f-a13b-cc26043cc3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:45.743900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1b559-7cd0-485f-a13b-cc26043cc3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:45.744191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1b559-7cd0-485f-a13b-cc26043cc3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:45.744474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1b559-7cd0-485f-a13b-cc26043cc3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:45.745712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1b559-7cd0-485f-a13b-cc26043cc3f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:45.746035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1b559-7cd0-485f-a13b-cc26043cc3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:45.747738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1b559-7cd0-485f-a13b-cc26043cc3f9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:45.758305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 220/437] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:50:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:45.763610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1b559-7cd0-485f-a13b-cc26043cc3f9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:45.778748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc02565-a016-4539-9b86-119aa657f8cb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:45.822428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553ad9d-6489-46a9-b70a-f91c00501fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:45.823586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ad9d-6489-46a9-b70a-f91c00501fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:45.823880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ad9d-6489-46a9-b70a-f91c00501fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:45.824158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553ad9d-6489-46a9-b70a-f91c00501fa6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:45.837469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553ad9d-6489-46a9-b70a-f91c00501fa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2553ad9d-6489-46a9-b70a-f91c00501fa6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:45.847920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:45.849102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:45.849911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 218/438] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:45.857960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553ad9d-6489-46a9-b70a-f91c00501fa6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:45.885988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:45.894745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1b559-7cd0-485f-a13b-cc26043cc3f9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:45.935222 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d5ddc-573e-40e9-be4d-ccfbe00fcb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:45.935871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d5ddc-573e-40e9-be4d-ccfbe00fcb40 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:45.936159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d5ddc-573e-40e9-be4d-ccfbe00fcb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:45.936429 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d5ddc-573e-40e9-be4d-ccfbe00fcb40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:45.977658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553ad9d-6489-46a9-b70a-f91c00501fa6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:45.979721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:46.008515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d5ddc-573e-40e9-be4d-ccfbe00fcb40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:46.010004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d5ddc-573e-40e9-be4d-ccfbe00fcb40 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:46.012812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a0133-10f3-48c2-8980-d334d7ba834f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:46.013638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a0133-10f3-48c2-8980-d334d7ba834f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.014234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a0133-10f3-48c2-8980-d334d7ba834f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.014517 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a0133-10f3-48c2-8980-d334d7ba834f None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.015269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a0133-10f3-48c2-8980-d334d7ba834f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:46.015583 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a0133-10f3-48c2-8980-d334d7ba834f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:46.017009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a0133-10f3-48c2-8980-d334d7ba834f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:46.025497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 219/439] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:46.026626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09421187-239d-4a5b-b621-c76198119430 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.026916 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09421187-239d-4a5b-b621-c76198119430 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.027196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09421187-239d-4a5b-b621-c76198119430 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.028409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 220/440] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:46.030713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09421187-239d-4a5b-b621-c76198119430 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:46.104181 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:46.111310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 221/441] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:50:46.112581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e72209-1295-4d64-be7a-254da5f3fa35 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.115165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e72209-1295-4d64-be7a-254da5f3fa35 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.115165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e72209-1295-4d64-be7a-254da5f3fa35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.130267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f6e72209-1295-4d64-be7a-254da5f3fa35 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:46.131449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f6e72209-1295-4d64-be7a-254da5f3fa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:46.132207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09421187-239d-4a5b-b621-c76198119430 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:46.165262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349864a3-5bac-4459-b9df-a26da85469e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:46.165901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349864a3-5bac-4459-b9df-a26da85469e4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.166185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349864a3-5bac-4459-b9df-a26da85469e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.166462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349864a3-5bac-4459-b9df-a26da85469e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.196733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:46.204954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 222/442] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:46.204954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e72209-1295-4d64-be7a-254da5f3fa35 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:46.246153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349864a3-5bac-4459-b9df-a26da85469e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:46.247607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349864a3-5bac-4459-b9df-a26da85469e4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:46.307049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e72209-1295-4d64-be7a-254da5f3fa35 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:46.329804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:46.340692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d724d3-28fe-4ca9-9ba0-d276af82896d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:46.341572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d724d3-28fe-4ca9-9ba0-d276af82896d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.341860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d724d3-28fe-4ca9-9ba0-d276af82896d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.342139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d724d3-28fe-4ca9-9ba0-d276af82896d None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.343437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d724d3-28fe-4ca9-9ba0-d276af82896d None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:46.344935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d724d3-28fe-4ca9-9ba0-d276af82896d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:46.346628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 223/443] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:46 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:46.348035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d724d3-28fe-4ca9-9ba0-d276af82896d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:46.348968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 221/444] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:50:46.352865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349864a3-5bac-4459-b9df-a26da85469e4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:46.414183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:46.445262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d724d3-28fe-4ca9-9ba0-d276af82896d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:46.452729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349864a3-5bac-4459-b9df-a26da85469e4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:46.478811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c50352-96c5-4dd5-932f-e68795b178ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:46.479594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c50352-96c5-4dd5-932f-e68795b178ac None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.479878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c50352-96c5-4dd5-932f-e68795b178ac None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.480154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c50352-96c5-4dd5-932f-e68795b178ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.485614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a87dbb-8274-4548-9325-b39ce76cde1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:46.486454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a87dbb-8274-4548-9325-b39ce76cde1f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.486739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a87dbb-8274-4548-9325-b39ce76cde1f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.487015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a87dbb-8274-4548-9325-b39ce76cde1f None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.488209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a87dbb-8274-4548-9325-b39ce76cde1f None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:46.489687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a87dbb-8274-4548-9325-b39ce76cde1f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:46.490627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c50352-96c5-4dd5-932f-e68795b178ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:46.491288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 222/445] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:46 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:50:46.496733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a87dbb-8274-4548-9325-b39ce76cde1f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-02c50352-96c5-4dd5-932f-e68795b178ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:46.498251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:46.499089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:46.499871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 224/446] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:50:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:46.500507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c50352-96c5-4dd5-932f-e68795b178ac None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:46.564241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:46.606180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a87dbb-8274-4548-9325-b39ce76cde1f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:46.610960 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c50352-96c5-4dd5-932f-e68795b178ac None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:46.640149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:46.647020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275f4d8-c326-4830-be44-c0fc735e2477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:46.653546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a30033-c7c8-4732-bd5b-6b76c0d7ba1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:46.653676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275f4d8-c326-4830-be44-c0fc735e2477 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.653676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275f4d8-c326-4830-be44-c0fc735e2477 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.653676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275f4d8-c326-4830-be44-c0fc735e2477 None admin] PATH_INFO: `/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.653897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a30033-c7c8-4732-bd5b-6b76c0d7ba1e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.653897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a30033-c7c8-4732-bd5b-6b76c0d7ba1e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.653897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a30033-c7c8-4732-bd5b-6b76c0d7ba1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.654065 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275f4d8-c326-4830-be44-c0fc735e2477 None admin] RBAC: Authorizing `identity:get_project(project_id=ca8a83413aa348d7a1e9fcd9b258330b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:46.654128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a30033-c7c8-4732-bd5b-6b76c0d7ba1e None admin] RBAC: Adding query filter params (group.id=803eb07f8d704ccfb6c6a179b7510dbd, role.id=61e62f29eafd45c8b9fb99205a60bc4e, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:46.654128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a30033-c7c8-4732-bd5b-6b76c0d7ba1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:46.654252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275f4d8-c326-4830-be44-c0fc735e2477 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:46.654315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a30033-c7c8-4732-bd5b-6b76c0d7ba1e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:46.654377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 223/447] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:46 2022] GET /identity/v3/projects/ca8a83413aa348d7a1e9fcd9b258330b => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:46.654377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275f4d8-c326-4830-be44-c0fc735e2477 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:46.659393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 225/448] 10.222.0.45 () {56 vars in 1376 bytes} [Tue Jul 5 15:50:46 2022] GET /identity/v3/role_assignments?group.id=803eb07f8d704ccfb6c6a179b7510dbd&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=61e62f29eafd45c8b9fb99205a60bc4e => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:46.660224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a30033-c7c8-4732-bd5b-6b76c0d7ba1e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:46.789599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a30033-c7c8-4732-bd5b-6b76c0d7ba1e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:46.790245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275f4d8-c326-4830-be44-c0fc735e2477 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:46.813274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:46.834185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d1c5d-c5fb-4709-97dc-c9c53df094a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:46.835125 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d1c5d-c5fb-4709-97dc-c9c53df094a3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.835476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d1c5d-c5fb-4709-97dc-c9c53df094a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.835743 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d1c5d-c5fb-4709-97dc-c9c53df094a3 None admin] PATH_INFO: `/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.837055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d1c5d-c5fb-4709-97dc-c9c53df094a3 None admin] RBAC: Authorizing `identity:get_group(group_id=803eb07f8d704ccfb6c6a179b7510dbd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:46.838552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d1c5d-c5fb-4709-97dc-c9c53df094a3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:46.840329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 226/449] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:46 2022] GET /identity/v3/groups/803eb07f8d704ccfb6c6a179b7510dbd => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:46.844265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a4738-8fbd-4a37-a86d-9feaf6dd5e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:46.845221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a4738-8fbd-4a37-a86d-9feaf6dd5e4c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:46.845632 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a4738-8fbd-4a37-a86d-9feaf6dd5e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:46.846102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a4738-8fbd-4a37-a86d-9feaf6dd5e4c None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:46.846319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d1c5d-c5fb-4709-97dc-c9c53df094a3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:46.848398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a4738-8fbd-4a37-a86d-9feaf6dd5e4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:46.849617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a4738-8fbd-4a37-a86d-9feaf6dd5e4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:46.853276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a4738-8fbd-4a37-a86d-9feaf6dd5e4c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:46.865575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 224/450] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:46.873504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a4738-8fbd-4a37-a86d-9feaf6dd5e4c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:46.956458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:46.974617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d1c5d-c5fb-4709-97dc-c9c53df094a3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:47.005193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a4738-8fbd-4a37-a86d-9feaf6dd5e4c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:47.022414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69d030-8e80-45a5-b22d-fdf3342a3c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:47.022414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d030-8e80-45a5-b22d-fdf3342a3c25 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:47.022725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d030-8e80-45a5-b22d-fdf3342a3c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:47.022923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d030-8e80-45a5-b22d-fdf3342a3c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:47.023775 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69d030-8e80-45a5-b22d-fdf3342a3c25 None admin] RBAC: Adding query filter params (group.id=803eb07f8d704ccfb6c6a179b7510dbd, role.id=2991af796dcd45e2b86208339339b977, scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:47.024095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69d030-8e80-45a5-b22d-fdf3342a3c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:47.026527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69d030-8e80-45a5-b22d-fdf3342a3c25 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:47.032064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 227/451] 10.222.0.45 () {56 vars in 1376 bytes} [Tue Jul 5 15:50:46 2022] GET /identity/v3/role_assignments?group.id=803eb07f8d704ccfb6c6a179b7510dbd&scope.project.id=ca8a83413aa348d7a1e9fcd9b258330b&role.id=2991af796dcd45e2b86208339339b977 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:47.054399 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccbe6d-6a58-401d-ba7b-a465c6002312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:47.055009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccbe6d-6a58-401d-ba7b-a465c6002312 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:47.055298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccbe6d-6a58-401d-ba7b-a465c6002312 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:47.055578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccbe6d-6a58-401d-ba7b-a465c6002312 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:47.138991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccbe6d-6a58-401d-ba7b-a465c6002312 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:47.140464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccbe6d-6a58-401d-ba7b-a465c6002312 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:47.206922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:47.248577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 225/452] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:47.254944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69d030-8e80-45a5-b22d-fdf3342a3c25 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:47.317850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:47.375686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69d030-8e80-45a5-b22d-fdf3342a3c25 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:47.408447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffbb0-5d63-48db-bd51-4c599dfe7938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:47.409323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffbb0-5d63-48db-bd51-4c599dfe7938 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:47.409605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffbb0-5d63-48db-bd51-4c599dfe7938 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:47.410510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffbb0-5d63-48db-bd51-4c599dfe7938 None admin] PATH_INFO: `/v3/projects/35018c3fbb4f440c9fd930631a71e6c6` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:47.411703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edffbb0-5d63-48db-bd51-4c599dfe7938 None admin] RBAC: Authorizing `identity:get_project(project_id=35018c3fbb4f440c9fd930631a71e6c6)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:47.413181 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edffbb0-5d63-48db-bd51-4c599dfe7938 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:47.414853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 228/453] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:50:47 2022] GET /identity/v3/projects/35018c3fbb4f440c9fd930631a71e6c6 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:47.420359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccbe6d-6a58-401d-ba7b-a465c6002312 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:47.516023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccbe6d-6a58-401d-ba7b-a465c6002312 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:47.548717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80dc1b-5b4f-4403-9cc5-892b9779a8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:47.549580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80dc1b-5b4f-4403-9cc5-892b9779a8dd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:47.549866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80dc1b-5b4f-4403-9cc5-892b9779a8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:47.550144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80dc1b-5b4f-4403-9cc5-892b9779a8dd None admin] PATH_INFO: `/v3/groups/d70ff95ce3524ded8aede53e07be5534` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:47.551849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80dc1b-5b4f-4403-9cc5-892b9779a8dd None admin] RBAC: Authorizing `identity:get_group(group_id=d70ff95ce3524ded8aede53e07be5534)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:47.553313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80dc1b-5b4f-4403-9cc5-892b9779a8dd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:47.554879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 226/454] 10.222.0.45 () {56 vars in 1180 bytes} [Tue Jul 5 15:50:47 2022] GET /identity/v3/groups/d70ff95ce3524ded8aede53e07be5534 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:47.560387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffbb0-5d63-48db-bd51-4c599dfe7938 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:47.654848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffbb0-5d63-48db-bd51-4c599dfe7938 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:47.670382 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:47.687719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ca4bb-485e-4b6f-9609-6147720c22b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:47.688356 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ca4bb-485e-4b6f-9609-6147720c22b5 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:47.688640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ca4bb-485e-4b6f-9609-6147720c22b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:47.688945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ca4bb-485e-4b6f-9609-6147720c22b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:47.689767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ca4bb-485e-4b6f-9609-6147720c22b5 None admin] RBAC: Adding query filter params (group.id=d70ff95ce3524ded8aede53e07be5534, role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=35018c3fbb4f440c9fd930631a71e6c6) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:47.690081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ca4bb-485e-4b6f-9609-6147720c22b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:47.691615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ca4bb-485e-4b6f-9609-6147720c22b5 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:47.697770 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 229/455] 10.222.0.45 () {56 vars in 1376 bytes} [Tue Jul 5 15:50:47 2022] GET /identity/v3/role_assignments?group.id=d70ff95ce3524ded8aede53e07be5534&scope.project.id=35018c3fbb4f440c9fd930631a71e6c6&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:50.094571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b76f05-0259-4193-8a3c-5b582d347b80 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:50.095085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b76f05-0259-4193-8a3c-5b582d347b80 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:50.095160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b76f05-0259-4193-8a3c-5b582d347b80 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:50.096724 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 227/456] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:50.105020 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fd560-a7dd-4194-b488-90a01ca156b0 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:50.105835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fd560-a7dd-4194-b488-90a01ca156b0 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:50.106311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fd560-a7dd-4194-b488-90a01ca156b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:50.130787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-ee2fd560-a7dd-4194-b488-90a01ca156b0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:50.132308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ee2fd560-a7dd-4194-b488-90a01ca156b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:50.227367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 230/457] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:50.266866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b76f05-0259-4193-8a3c-5b582d347b80 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:50.307690 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:50.414399 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b76f05-0259-4193-8a3c-5b582d347b80 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:50.445701 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab0804-4456-4c89-b360-0d4530d798d7 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:50.446213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab0804-4456-4c89-b360-0d4530d798d7 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:50.446643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab0804-4456-4c89-b360-0d4530d798d7 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:50.448249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 231/458] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:50.461470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4a8cc-7f23-43fe-8d3a-dcd97efa3214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:50.462152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99096-c5c5-486b-b1ab-7fa7c78d99e6 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:50.462269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4a8cc-7f23-43fe-8d3a-dcd97efa3214 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:50.462540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4a8cc-7f23-43fe-8d3a-dcd97efa3214 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:50.462815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4a8cc-7f23-43fe-8d3a-dcd97efa3214 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:50.462975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99096-c5c5-486b-b1ab-7fa7c78d99e6 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:50.463396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99096-c5c5-486b-b1ab-7fa7c78d99e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:50.477495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4a8cc-7f23-43fe-8d3a-dcd97efa3214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:50.478976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4a8cc-7f23-43fe-8d3a-dcd97efa3214 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:50.497292 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-00d99096-c5c5-486b-b1ab-7fa7c78d99e6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:50.498476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-00d99096-c5c5-486b-b1ab-7fa7c78d99e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-39c4a8cc-7f23-43fe-8d3a-dcd97efa3214 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:50.517912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:50.518824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:50.520246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:50.520246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:50.520246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:50.520246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:50.520246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 228/459] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:50.524807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52188be-1da0-4c66-a1b9-80cfc59d9136 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:50.524807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52188be-1da0-4c66-a1b9-80cfc59d9136 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:50.524807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52188be-1da0-4c66-a1b9-80cfc59d9136 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:50.524807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 229/460] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:50.525966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52188be-1da0-4c66-a1b9-80cfc59d9136 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:50.588885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 232/461] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 05 15:50:50.590666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360533e-f54b-4c78-a9ef-23e287b7e704 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:50.591209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360533e-f54b-4c78-a9ef-23e287b7e704 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:50.591737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360533e-f54b-4c78-a9ef-23e287b7e704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:50.615017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-2360533e-f54b-4c78-a9ef-23e287b7e704 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:50.624787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2360533e-f54b-4c78-a9ef-23e287b7e704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:50.625247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:50.640042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52188be-1da0-4c66-a1b9-80cfc59d9136 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:50.670771 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:50.674450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047dc8e-6d98-443b-b2b7-aab0d83abb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:50.675142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047dc8e-6d98-443b-b2b7-aab0d83abb0a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:50.675968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047dc8e-6d98-443b-b2b7-aab0d83abb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:50.676247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047dc8e-6d98-443b-b2b7-aab0d83abb0a None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:50.677012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047dc8e-6d98-443b-b2b7-aab0d83abb0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:50.677318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047dc8e-6d98-443b-b2b7-aab0d83abb0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:50.678712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047dc8e-6d98-443b-b2b7-aab0d83abb0a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:50.697387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 230/462] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:50.698433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047dc8e-6d98-443b-b2b7-aab0d83abb0a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:50.717419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 233/463] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:50.725161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360533e-f54b-4c78-a9ef-23e287b7e704 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:50.851418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047dc8e-6d98-443b-b2b7-aab0d83abb0a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:50.865158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360533e-f54b-4c78-a9ef-23e287b7e704 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:50.904016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554eea1-f2cf-49d3-8553-366bd48cbbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:50.905200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554eea1-f2cf-49d3-8553-366bd48cbbc9 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:50.905368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511cff8-ba0d-4d1b-a8db-3f4cee508534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:50.905772 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554eea1-f2cf-49d3-8553-366bd48cbbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:50.906246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554eea1-f2cf-49d3-8553-366bd48cbbc9 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:50.906371 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511cff8-ba0d-4d1b-a8db-3f4cee508534 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:50.906811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511cff8-ba0d-4d1b-a8db-3f4cee508534 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:50.907231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511cff8-ba0d-4d1b-a8db-3f4cee508534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:50.907389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554eea1-f2cf-49d3-8553-366bd48cbbc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:50.909269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554eea1-f2cf-49d3-8553-366bd48cbbc9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:50.921677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511cff8-ba0d-4d1b-a8db-3f4cee508534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:50.923124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511cff8-ba0d-4d1b-a8db-3f4cee508534 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:50.925023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b511cff8-ba0d-4d1b-a8db-3f4cee508534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:50.942184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:50.943142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:50.943998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:50.943998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:50.943998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:50.943998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:50.943998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 231/464] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:50.949022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511cff8-ba0d-4d1b-a8db-3f4cee508534 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:51.016442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 234/465] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:50 2022] POST /identity/v3/users => generated 246 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:51.017446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554eea1-f2cf-49d3-8553-366bd48cbbc9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:51.061696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:51.066398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511cff8-ba0d-4d1b-a8db-3f4cee508534 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:51.114610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081401e2-2a28-41f3-a28d-dd4c9d72b78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:51.116188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081401e2-2a28-41f3-a28d-dd4c9d72b78a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:51.116682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081401e2-2a28-41f3-a28d-dd4c9d72b78a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:51.117178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081401e2-2a28-41f3-a28d-dd4c9d72b78a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:51.129884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081401e2-2a28-41f3-a28d-dd4c9d72b78a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:51.131683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081401e2-2a28-41f3-a28d-dd4c9d72b78a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-081401e2-2a28-41f3-a28d-dd4c9d72b78a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:51.153305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:51.154207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:51.155046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:51.155046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:51.155046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:51.155046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:51.157181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 232/466] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:51.159035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b093f-94b3-4880-91a9-aeb18df10492 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:51.159595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b093f-94b3-4880-91a9-aeb18df10492 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:51.160086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b093f-94b3-4880-91a9-aeb18df10492 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:51.162125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 233/467] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:51.163262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b093f-94b3-4880-91a9-aeb18df10492 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:51.165703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554eea1-f2cf-49d3-8553-366bd48cbbc9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:51.212867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf333767-44dc-4393-96ec-72ed45da5dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:51.213921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf333767-44dc-4393-96ec-72ed45da5dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:51.214206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf333767-44dc-4393-96ec-72ed45da5dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:51.214483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf333767-44dc-4393-96ec-72ed45da5dd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:51.215707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf333767-44dc-4393-96ec-72ed45da5dd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:51.216021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf333767-44dc-4393-96ec-72ed45da5dd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:51.217786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf333767-44dc-4393-96ec-72ed45da5dd8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:51.222901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:51.234537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 235/468] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:51.235996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e09e9-bf0e-4776-8cc7-cb68b1af9694 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:51.236430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e09e9-bf0e-4776-8cc7-cb68b1af9694 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:51.236995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e09e9-bf0e-4776-8cc7-cb68b1af9694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:51.255227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b64e09e9-bf0e-4776-8cc7-cb68b1af9694 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:51.256723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b64e09e9-bf0e-4776-8cc7-cb68b1af9694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:51.304336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b093f-94b3-4880-91a9-aeb18df10492 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:51.342003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0ae2-7a44-436c-a888-a599bc0cab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:51.343037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0ae2-7a44-436c-a888-a599bc0cab5e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:51.343525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0ae2-7a44-436c-a888-a599bc0cab5e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:51.344000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f0ae2-7a44-436c-a888-a599bc0cab5e None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:51.344997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0ae2-7a44-436c-a888-a599bc0cab5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:51.345508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0ae2-7a44-436c-a888-a599bc0cab5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:51.347150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f0ae2-7a44-436c-a888-a599bc0cab5e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:51.349197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 236/469] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:51.350090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e09e9-bf0e-4776-8cc7-cb68b1af9694 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:51.360293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 234/470] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:51.365806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0ae2-7a44-436c-a888-a599bc0cab5e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:51.370566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:51.478611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e09e9-bf0e-4776-8cc7-cb68b1af9694 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:51.485973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f0ae2-7a44-436c-a888-a599bc0cab5e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:51.494677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:51.517648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ff803-2b84-4590-b6a3-4c51d9e65b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:51.518319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff803-2b84-4590-b6a3-4c51d9e65b5d None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:51.518612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff803-2b84-4590-b6a3-4c51d9e65b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:51.518916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff803-2b84-4590-b6a3-4c51d9e65b5d None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:51.519612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ff803-2b84-4590-b6a3-4c51d9e65b5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:51.521122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ff803-2b84-4590-b6a3-4c51d9e65b5d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:51.522901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0574bf-1c47-4243-bae9-1fc8d34dba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:51.523567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0574bf-1c47-4243-bae9-1fc8d34dba54 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:51.523853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0574bf-1c47-4243-bae9-1fc8d34dba54 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:51.524131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0574bf-1c47-4243-bae9-1fc8d34dba54 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:51.524859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0574bf-1c47-4243-bae9-1fc8d34dba54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:51.526354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0574bf-1c47-4243-bae9-1fc8d34dba54 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:51.627236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 237/471] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:51 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:51.628068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ff803-2b84-4590-b6a3-4c51d9e65b5d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:51.654329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 235/472] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:51 2022] POST /identity/v3/users => generated 248 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:50:51.663607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc6f0d-7a4b-4be3-9966-5c075ec612b3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:51.663952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc6f0d-7a4b-4be3-9966-5c075ec612b3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:51.664168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc6f0d-7a4b-4be3-9966-5c075ec612b3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:51.670138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 236/473] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:51 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:51.674361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfc644-3921-43ee-8b66-78644d69f2a9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:51.675663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfc644-3921-43ee-8b66-78644d69f2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:51.677581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfc644-3921-43ee-8b66-78644d69f2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:51.715947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-48dfc644-3921-43ee-8b66-78644d69f2a9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:51.718535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-48dfc644-3921-43ee-8b66-78644d69f2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:51.777425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:51.817125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 237/474] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:51.828074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ff803-2b84-4590-b6a3-4c51d9e65b5d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:51.844649 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfc644-3921-43ee-8b66-78644d69f2a9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:51.871920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3cf2d-fa98-4946-a534-14447db187d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:51.872725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3cf2d-fa98-4946-a534-14447db187d4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:51.873096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3cf2d-fa98-4946-a534-14447db187d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:51.873372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3cf2d-fa98-4946-a534-14447db187d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:51.884822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3cf2d-fa98-4946-a534-14447db187d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:51.886335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3cf2d-fa98-4946-a534-14447db187d4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-aee3cf2d-fa98-4946-a534-14447db187d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:51.903791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:51.904658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:51.905555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:51.905555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:51.905555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:51.905555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:51.905555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 238/475] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:51.911149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3cf2d-fa98-4946-a534-14447db187d4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:52.013008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:52.032606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfc644-3921-43ee-8b66-78644d69f2a9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:52.057918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3cf2d-fa98-4946-a534-14447db187d4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:52.074386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b11b4-5d14-4e9f-bc1f-4893f8f1ef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:52.075443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b11b4-5d14-4e9f-bc1f-4893f8f1ef3d None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:52.075788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b11b4-5d14-4e9f-bc1f-4893f8f1ef3d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:52.076520 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b11b4-5d14-4e9f-bc1f-4893f8f1ef3d None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:52.077539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b11b4-5d14-4e9f-bc1f-4893f8f1ef3d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:52.079606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b11b4-5d14-4e9f-bc1f-4893f8f1ef3d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:52.083159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:52.098475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ffc3a9-678e-4356-8d04-65c1c6a136c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:52.099202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffc3a9-678e-4356-8d04-65c1c6a136c8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:52.099488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffc3a9-678e-4356-8d04-65c1c6a136c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:52.101316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffc3a9-678e-4356-8d04-65c1c6a136c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:52.101316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ffc3a9-678e-4356-8d04-65c1c6a136c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:52.101316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ffc3a9-678e-4356-8d04-65c1c6a136c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:52.103023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ffc3a9-678e-4356-8d04-65c1c6a136c8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:52.116224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 239/476] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:52.121766 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ffc3a9-678e-4356-8d04-65c1c6a136c8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:52.128138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.sql.core [None req-b62b11b4-5d14-4e9f-bc1f-4893f8f1ef3d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 05 15:50:52.128138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 05 15:50:52.128138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [parameters: {'id': 'ecbf514cf6da46c1aefc2ec779e32ae3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 05 15:50:52.128138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80398) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b62b11b4-5d14-4e9f-bc1f-4893f8f1ef3d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result.read() Jul 05 15:50:52.154726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return ref.to_dict() Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:50:52.167618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application yield resource Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.session.rollback() Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.force_reraise() Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise self.value Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application session.commit() Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 05 15:50:52.168694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._prepare_impl() Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.session.flush() Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._flush(objects) Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application compat.raise_( Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application flush_context.execute() Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rec.execute(self) Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 05 15:50:52.169566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:50:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result.read() Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:50:52.171232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [parameters: {'id': 'ecbf514cf6da46c1aefc2ec779e32ae3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:52.176800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 05 15:50:52.177835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:52.178652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 238/477] 10.222.0.45 () {60 vars in 1131 bytes} [Tue Jul 5 15:50:51 2022] POST /identity/v3/roles => generated 144 bytes in 317 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:52.178652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b11b4-5d14-4e9f-bc1f-4893f8f1ef3d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:52.233254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ffc3a9-678e-4356-8d04-65c1c6a136c8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:52.291882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c1a09-a86e-4a84-8dea-8f5db849fdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:52.292820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c1a09-a86e-4a84-8dea-8f5db849fdd7 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:52.293394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c1a09-a86e-4a84-8dea-8f5db849fdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:52.300793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c1a09-a86e-4a84-8dea-8f5db849fdd7 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:52.300793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c1a09-a86e-4a84-8dea-8f5db849fdd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:52.300793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c1a09-a86e-4a84-8dea-8f5db849fdd7 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:52.359078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b11b4-5d14-4e9f-bc1f-4893f8f1ef3d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:52.390690 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 240/478] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:52 2022] POST /identity/v3/users => generated 249 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:52.403620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a842ac-be8e-430b-a43e-804e0d87db28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:52.404491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a842ac-be8e-430b-a43e-804e0d87db28 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:52.404854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a842ac-be8e-430b-a43e-804e0d87db28 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:52.405222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a842ac-be8e-430b-a43e-804e0d87db28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:52.418517 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a842ac-be8e-430b-a43e-804e0d87db28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a1a842ac-be8e-430b-a43e-804e0d87db28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:52.427620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:52.428551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:50:52.429361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:52.432948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 239/479] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:50:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:52.435636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c1a09-a86e-4a84-8dea-8f5db849fdd7 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:52.483816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:52.550820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c1a09-a86e-4a84-8dea-8f5db849fdd7 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:52.589333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1db21-bdd8-4432-aa7b-9e82e193e86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:52.590057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1db21-bdd8-4432-aa7b-9e82e193e86b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:52.590347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1db21-bdd8-4432-aa7b-9e82e193e86b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:52.590628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1db21-bdd8-4432-aa7b-9e82e193e86b None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:52.591394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1db21-bdd8-4432-aa7b-9e82e193e86b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:52.591705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1db21-bdd8-4432-aa7b-9e82e193e86b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:52.593104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1db21-bdd8-4432-aa7b-9e82e193e86b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:52.608962 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 241/480] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:53.356280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9fa3d-88d8-4186-9676-a58914e1163d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:53.356817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9fa3d-88d8-4186-9676-a58914e1163d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:53.356894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9fa3d-88d8-4186-9676-a58914e1163d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:53.358740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 240/481] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:53.363820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872a1b7-4dab-4c16-bab0-1b45eb568376 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:53.364224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872a1b7-4dab-4c16-bab0-1b45eb568376 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:53.364410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872a1b7-4dab-4c16-bab0-1b45eb568376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:53.385770 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3872a1b7-4dab-4c16-bab0-1b45eb568376 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:53.386982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3872a1b7-4dab-4c16-bab0-1b45eb568376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:53.470105 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 242/482] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:53.505961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9fa3d-88d8-4186-9676-a58914e1163d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:53.529111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:53.645372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9fa3d-88d8-4186-9676-a58914e1163d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:53.692849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a080fa5-e391-487b-ab73-ee6017463803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:53.694017 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a080fa5-e391-487b-ab73-ee6017463803 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:53.694500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a080fa5-e391-487b-ab73-ee6017463803 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:53.694969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a080fa5-e391-487b-ab73-ee6017463803 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:53.716390 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a080fa5-e391-487b-ab73-ee6017463803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8a080fa5-e391-487b-ab73-ee6017463803 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:53.724394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:53.725294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:50:53.726175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:53.733093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 241/483] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:50:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:53.734964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872a1b7-4dab-4c16-bab0-1b45eb568376 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:53.767578 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:53.859260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872a1b7-4dab-4c16-bab0-1b45eb568376 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:53.906828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e14857-dde4-4eca-85a5-987806455c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:53.907417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e14857-dde4-4eca-85a5-987806455c9f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:53.907701 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e14857-dde4-4eca-85a5-987806455c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:53.907981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e14857-dde4-4eca-85a5-987806455c9f None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:53.908742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e14857-dde4-4eca-85a5-987806455c9f None admin] RBAC: Adding query filter params (name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:53.909072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e14857-dde4-4eca-85a5-987806455c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:53.910482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e14857-dde4-4eca-85a5-987806455c9f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:53.924107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 243/484] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:50:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:53.930292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a080fa5-e391-487b-ab73-ee6017463803 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:54.017503 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc63838-86a8-451d-8001-c16b12c799c7 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.017843 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc63838-86a8-451d-8001-c16b12c799c7 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.018068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc63838-86a8-451d-8001-c16b12c799c7 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.019692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 244/485] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:54.024866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c6c82-3113-49bd-a631-bafd8d252a40 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.025160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c6c82-3113-49bd-a631-bafd8d252a40 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.025441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c6c82-3113-49bd-a631-bafd8d252a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.034895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:54.049306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-cc6c6c82-3113-49bd-a631-bafd8d252a40 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:54.051036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-cc6c6c82-3113-49bd-a631-bafd8d252a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:54.077718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a080fa5-e391-487b-ab73-ee6017463803 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:54.142134 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b803bdf-aa11-4a1a-9ce5-484ec7911852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:54.142932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b803bdf-aa11-4a1a-9ce5-484ec7911852 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.143214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b803bdf-aa11-4a1a-9ce5-484ec7911852 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.143490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b803bdf-aa11-4a1a-9ce5-484ec7911852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.166656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b803bdf-aa11-4a1a-9ce5-484ec7911852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:54.168114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b803bdf-aa11-4a1a-9ce5-484ec7911852 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:54.180942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 245/486] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4b803bdf-aa11-4a1a-9ce5-484ec7911852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:54.190528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:54.191393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:54.192223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:54.192223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:54.192223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:54.192223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:54.192223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 242/487] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:54.198008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c6c82-3113-49bd-a631-bafd8d252a40 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:54.203481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b803bdf-aa11-4a1a-9ce5-484ec7911852 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:54.250097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:54.356812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c6c82-3113-49bd-a631-bafd8d252a40 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:54.377624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b803bdf-aa11-4a1a-9ce5-484ec7911852 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:54.422588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425d61f-62e3-44ee-996f-ae5feaad4bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:54.423165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425d61f-62e3-44ee-996f-ae5feaad4bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.423609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425d61f-62e3-44ee-996f-ae5feaad4bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.424028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425d61f-62e3-44ee-996f-ae5feaad4bd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.424997 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425d61f-62e3-44ee-996f-ae5feaad4bd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:54.425943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425d61f-62e3-44ee-996f-ae5feaad4bd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:54.427456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425d61f-62e3-44ee-996f-ae5feaad4bd2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:54.437126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:54.443943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c26e4-8503-442f-99a4-26b4bf5c3f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:54.444745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c26e4-8503-442f-99a4-26b4bf5c3f30 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.447119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c26e4-8503-442f-99a4-26b4bf5c3f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.447119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c26e4-8503-442f-99a4-26b4bf5c3f30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.450228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 246/488] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:54.453305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425d61f-62e3-44ee-996f-ae5feaad4bd2 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:54.468511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c26e4-8503-442f-99a4-26b4bf5c3f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-624c26e4-8503-442f-99a4-26b4bf5c3f30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:54.477352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:54.478388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:50:54.479372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:54.483476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 243/489] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:50:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:54.488985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf45725-4cde-40b4-a5c8-3d7bc7b0e652 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.489441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf45725-4cde-40b4-a5c8-3d7bc7b0e652 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.489886 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf45725-4cde-40b4-a5c8-3d7bc7b0e652 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.491578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 244/490] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:54.493297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf45725-4cde-40b4-a5c8-3d7bc7b0e652 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:54.564414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:54.580501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425d61f-62e3-44ee-996f-ae5feaad4bd2 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:54.618083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf45725-4cde-40b4-a5c8-3d7bc7b0e652 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:54.618988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09477b1e-fcba-43d5-b942-2f3e66e4a39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:54.620042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09477b1e-fcba-43d5-b942-2f3e66e4a39d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.620571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09477b1e-fcba-43d5-b942-2f3e66e4a39d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.621006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09477b1e-fcba-43d5-b942-2f3e66e4a39d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.628714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09477b1e-fcba-43d5-b942-2f3e66e4a39d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:54.630470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09477b1e-fcba-43d5-b942-2f3e66e4a39d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-09477b1e-fcba-43d5-b942-2f3e66e4a39d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:50:54.648060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:54.649205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:54.650155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:54.650155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:54.650155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:50:54.650155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:50:54.650155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:50:54.650155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:50:54.650155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:50:54.650155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:50:54.650155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:50:54.650155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:54.652347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 247/491] 10.222.0.45 () {56 vars in 1093 bytes} [Tue Jul 5 15:50:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:54.654118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643b8e1-274f-45b6-83a4-8aaef9bd4dda None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.654602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643b8e1-274f-45b6-83a4-8aaef9bd4dda None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.655069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643b8e1-274f-45b6-83a4-8aaef9bd4dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.655885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4335a8-fca3-42b4-a4b9-e2481a75dbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:54.656595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4335a8-fca3-42b4-a4b9-e2481a75dbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.657184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4335a8-fca3-42b4-a4b9-e2481a75dbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.657473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4335a8-fca3-42b4-a4b9-e2481a75dbe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.658233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4335a8-fca3-42b4-a4b9-e2481a75dbe0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:54.658543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4335a8-fca3-42b4-a4b9-e2481a75dbe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:54.660621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4335a8-fca3-42b4-a4b9-e2481a75dbe0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:54.672130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 245/492] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:50:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:54.673553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4335a8-fca3-42b4-a4b9-e2481a75dbe0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:54.682004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4643b8e1-274f-45b6-83a4-8aaef9bd4dda None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:54.683533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4643b8e1-274f-45b6-83a4-8aaef9bd4dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:54.738406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:54.765623 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 248/493] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:54.767856 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643b8e1-274f-45b6-83a4-8aaef9bd4dda None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:54.816624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4335a8-fca3-42b4-a4b9-e2481a75dbe0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:54.860931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db4f8d-28ae-4bb6-982a-13ff3876e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:54.861958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db4f8d-28ae-4bb6-982a-13ff3876e888 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.862470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db4f8d-28ae-4bb6-982a-13ff3876e888 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.863372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:54.864470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db4f8d-28ae-4bb6-982a-13ff3876e888 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.865571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db4f8d-28ae-4bb6-982a-13ff3876e888 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:54.866483 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db4f8d-28ae-4bb6-982a-13ff3876e888 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:54.868852 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db4f8d-28ae-4bb6-982a-13ff3876e888 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:54.889849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643b8e1-274f-45b6-83a4-8aaef9bd4dda None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:54.917720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 246/494] 10.222.0.45 () {56 vars in 1133 bytes} [Tue Jul 5 15:50:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:54.923098 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db4f8d-28ae-4bb6-982a-13ff3876e888 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:54.933545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe33b0-b595-402d-a489-62d8a73d853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:54.935092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe33b0-b595-402d-a489-62d8a73d853a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:54.935665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe33b0-b595-402d-a489-62d8a73d853a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:54.936499 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe33b0-b595-402d-a489-62d8a73d853a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:54.951131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe33b0-b595-402d-a489-62d8a73d853a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:54.953367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe33b0-b595-402d-a489-62d8a73d853a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-8cbe33b0-b595-402d-a489-62d8a73d853a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:54.978861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:54.979764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:54.980576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:54.980576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:54.980576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:54.980576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:54.980576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 249/495] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:54.981301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe33b0-b595-402d-a489-62d8a73d853a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:55.032199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db4f8d-28ae-4bb6-982a-13ff3876e888 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:55.043489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:55.069769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f469cee-484d-443c-aea8-c141aacb60d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:55.070559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f469cee-484d-443c-aea8-c141aacb60d5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.070839 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f469cee-484d-443c-aea8-c141aacb60d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.071112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f469cee-484d-443c-aea8-c141aacb60d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.082517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe33b0-b595-402d-a489-62d8a73d853a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:55.083105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f469cee-484d-443c-aea8-c141aacb60d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4f469cee-484d-443c-aea8-c141aacb60d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:55.091526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:55.092559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:50:55.093545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:55.095807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 247/496] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:50:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:55.097222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7434f-328c-48db-b5e9-f90a912abf99 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.097504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7434f-328c-48db-b5e9-f90a912abf99 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.097777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7434f-328c-48db-b5e9-f90a912abf99 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.099177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 248/497] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:55.101810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7434f-328c-48db-b5e9-f90a912abf99 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:55.116852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:55.127071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3595e-c89e-41d3-8e17-7142b8868c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:55.127742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3595e-c89e-41d3-8e17-7142b8868c8f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.128522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3595e-c89e-41d3-8e17-7142b8868c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.128825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3595e-c89e-41d3-8e17-7142b8868c8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.200829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3595e-c89e-41d3-8e17-7142b8868c8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:55.202467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3595e-c89e-41d3-8e17-7142b8868c8f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:55.211895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7434f-328c-48db-b5e9-f90a912abf99 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:55.248533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c810e-5892-4a8c-8609-d2265a06809a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:55.249225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c810e-5892-4a8c-8609-d2265a06809a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.249504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c810e-5892-4a8c-8609-d2265a06809a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.249783 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c810e-5892-4a8c-8609-d2265a06809a None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.250516 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c810e-5892-4a8c-8609-d2265a06809a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:55.250814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c810e-5892-4a8c-8609-d2265a06809a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:55.252190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c810e-5892-4a8c-8609-d2265a06809a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:55.264714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 249/498] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:55.265616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c810e-5892-4a8c-8609-d2265a06809a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:55.300914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:55.318524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 250/499] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:55.320053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8048e-49d6-4f73-982e-f7bd36560234 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.320340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8048e-49d6-4f73-982e-f7bd36560234 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.320618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8048e-49d6-4f73-982e-f7bd36560234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.338132 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-bee8048e-49d6-4f73-982e-f7bd36560234 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:55.339406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-bee8048e-49d6-4f73-982e-f7bd36560234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:55.360529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:55.379009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c810e-5892-4a8c-8609-d2265a06809a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:55.406680 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 251/500] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:55.407570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8048e-49d6-4f73-982e-f7bd36560234 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:55.411928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799bab5-b9b9-4962-a641-7c171929686f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:55.412606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799bab5-b9b9-4962-a641-7c171929686f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.412940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799bab5-b9b9-4962-a641-7c171929686f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.413222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799bab5-b9b9-4962-a641-7c171929686f None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.413978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799bab5-b9b9-4962-a641-7c171929686f None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:55.414289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799bab5-b9b9-4962-a641-7c171929686f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:55.415678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799bab5-b9b9-4962-a641-7c171929686f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:55.428708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 250/501] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:50:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:55.428708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799bab5-b9b9-4962-a641-7c171929686f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:55.509550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8048e-49d6-4f73-982e-f7bd36560234 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:55.514357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:55.523500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799bab5-b9b9-4962-a641-7c171929686f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:55.542277 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee21d7-28d2-489b-a6a9-ae21571c1f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:55.543090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee21d7-28d2-489b-a6a9-ae21571c1f16 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.543375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee21d7-28d2-489b-a6a9-ae21571c1f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.543654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee21d7-28d2-489b-a6a9-ae21571c1f16 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.551064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee21d7-28d2-489b-a6a9-ae21571c1f16 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:55.552490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee21d7-28d2-489b-a6a9-ae21571c1f16 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:55.557913 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c6eca-efc5-4f6f-a4f7-540ceb12933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:55.558686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c6eca-efc5-4f6f-a4f7-540ceb12933a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.558966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c6eca-efc5-4f6f-a4f7-540ceb12933a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.559238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c6eca-efc5-4f6f-a4f7-540ceb12933a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-5bee21d7-28d2-489b-a6a9-ae21571c1f16 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:50:55.567197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:55.568058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:55.569003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 252/502] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:55.569541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7753fbe-06e3-458e-9675-86edf2850895 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.569824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7753fbe-06e3-458e-9675-86edf2850895 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.570153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7753fbe-06e3-458e-9675-86edf2850895 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.571329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 253/503] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:55.572015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7753fbe-06e3-458e-9675-86edf2850895 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:55.574078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c6eca-efc5-4f6f-a4f7-540ceb12933a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:55.575685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c6eca-efc5-4f6f-a4f7-540ceb12933a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:55.589526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c78c6eca-efc5-4f6f-a4f7-540ceb12933a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:55.592963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:55.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:55.594667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:55.594667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:55.594667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:55.594667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:55.594667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 251/504] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:55.595597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c6eca-efc5-4f6f-a4f7-540ceb12933a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:55.675796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7753fbe-06e3-458e-9675-86edf2850895 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:55.693124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c6eca-efc5-4f6f-a4f7-540ceb12933a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:55.709710 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bf8e9-fa7e-466e-872c-42f2e6f32130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:55.710482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bf8e9-fa7e-466e-872c-42f2e6f32130 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.710761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bf8e9-fa7e-466e-872c-42f2e6f32130 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.711030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bf8e9-fa7e-466e-872c-42f2e6f32130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.721109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bf8e9-fa7e-466e-872c-42f2e6f32130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:55.723086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bf8e9-fa7e-466e-872c-42f2e6f32130 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:55.725636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417b228-40d9-4954-8ed0-9e4d04f8d0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:55.726407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417b228-40d9-4954-8ed0-9e4d04f8d0ec None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.726947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417b228-40d9-4954-8ed0-9e4d04f8d0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.727224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417b228-40d9-4954-8ed0-9e4d04f8d0ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.738023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417b228-40d9-4954-8ed0-9e4d04f8d0ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:55.739478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417b228-40d9-4954-8ed0-9e4d04f8d0ec None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:55.741204 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-791bf8e9-fa7e-466e-872c-42f2e6f32130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:55.741368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:55.742351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:55.743224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:55.743224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:55.743224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:55.743224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:55.743224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 254/505] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:55.743478 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bf8e9-fa7e-466e-872c-42f2e6f32130 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c417b228-40d9-4954-8ed0-9e4d04f8d0ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:55.756670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:55.766922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:55.767777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:55.767777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:55.767777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:55.767777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:55.767777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 252/506] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:55.767777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bfd01-6bb8-4194-a314-08b63ec37094 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.767777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bfd01-6bb8-4194-a314-08b63ec37094 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.767777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bfd01-6bb8-4194-a314-08b63ec37094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.776569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-432bfd01-6bb8-4194-a314-08b63ec37094 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:55.777791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-432bfd01-6bb8-4194-a314-08b63ec37094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:55.813049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:55.842669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 253/507] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:55.843710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bfd01-6bb8-4194-a314-08b63ec37094 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:55.850054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bf8e9-fa7e-466e-872c-42f2e6f32130 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:55.885400 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcbff0-2818-4ccc-9325-3d77f82598c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:55.886111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcbff0-2818-4ccc-9325-3d77f82598c3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.886393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcbff0-2818-4ccc-9325-3d77f82598c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.886666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcbff0-2818-4ccc-9325-3d77f82598c3 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.887512 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dcbff0-2818-4ccc-9325-3d77f82598c3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:55.887827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dcbff0-2818-4ccc-9325-3d77f82598c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:55.889297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dcbff0-2818-4ccc-9325-3d77f82598c3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:55.926102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 255/508] 10.222.0.45 () {56 vars in 1137 bytes} [Tue Jul 5 15:50:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:55.926916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcbff0-2818-4ccc-9325-3d77f82598c3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:55.943044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bfd01-6bb8-4194-a314-08b63ec37094 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:55.966187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:55.977316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a981a-ac01-42e9-9d8a-e44c94164b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:55.978031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a981a-ac01-42e9-9d8a-e44c94164b8c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:55.978318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a981a-ac01-42e9-9d8a-e44c94164b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:55.978596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a981a-ac01-42e9-9d8a-e44c94164b8c None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:55.979322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a981a-ac01-42e9-9d8a-e44c94164b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:55.979630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a981a-ac01-42e9-9d8a-e44c94164b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:55.981056 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a981a-ac01-42e9-9d8a-e44c94164b8c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:55.992169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 254/509] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:55.992970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a981a-ac01-42e9-9d8a-e44c94164b8c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.024576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcbff0-2818-4ccc-9325-3d77f82598c3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:56.050481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:56.058924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79edfc0-c862-4e20-815b-008b989147dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:56.063114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79edfc0-c862-4e20-815b-008b989147dc None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:56.063114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79edfc0-c862-4e20-815b-008b989147dc None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:56.063114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79edfc0-c862-4e20-815b-008b989147dc None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:56.063114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79edfc0-c862-4e20-815b-008b989147dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:56.063114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79edfc0-c862-4e20-815b-008b989147dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:56.063114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79edfc0-c862-4e20-815b-008b989147dc None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:56.074204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 256/510] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:56.075052 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79edfc0-c862-4e20-815b-008b989147dc None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.092362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a981a-ac01-42e9-9d8a-e44c94164b8c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:56.125588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5dcec-86de-44c4-b544-f4e8a4a1dd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:56.126526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5dcec-86de-44c4-b544-f4e8a4a1dd9c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:56.126810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5dcec-86de-44c4-b544-f4e8a4a1dd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:56.127086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5dcec-86de-44c4-b544-f4e8a4a1dd9c None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:56.127808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5dcec-86de-44c4-b544-f4e8a4a1dd9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:56.128665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5dcec-86de-44c4-b544-f4e8a4a1dd9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:56.130083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5dcec-86de-44c4-b544-f4e8a4a1dd9c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:56.141543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 255/511] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:56.142317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5dcec-86de-44c4-b544-f4e8a4a1dd9c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.178686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79edfc0-c862-4e20-815b-008b989147dc None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:56.185888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:56.211635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c922da-2337-4c1f-8ca4-228d91eaa8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:56.212408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c922da-2337-4c1f-8ca4-228d91eaa8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:56.212687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c922da-2337-4c1f-8ca4-228d91eaa8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:56.212987 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c922da-2337-4c1f-8ca4-228d91eaa8b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:56.222982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c922da-2337-4c1f-8ca4-228d91eaa8b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-e9c922da-2337-4c1f-8ca4-228d91eaa8b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:56.230814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:56.231688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:56.232493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 257/512] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:50:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:56.233172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c922da-2337-4c1f-8ca4-228d91eaa8b0 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.243574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5dcec-86de-44c4-b544-f4e8a4a1dd9c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:56.276721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d02ea5-001a-482d-b3ae-6377f0cde498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:56.277324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d02ea5-001a-482d-b3ae-6377f0cde498 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:56.277609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d02ea5-001a-482d-b3ae-6377f0cde498 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:56.277887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d02ea5-001a-482d-b3ae-6377f0cde498 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:56.285098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:56.338775 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c922da-2337-4c1f-8ca4-228d91eaa8b0 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:56.344023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d02ea5-001a-482d-b3ae-6377f0cde498 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:56.345518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d02ea5-001a-482d-b3ae-6377f0cde498 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:56.372282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940daaf3-3b09-41ad-8df9-0638fd61b643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:56.373099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940daaf3-3b09-41ad-8df9-0638fd61b643 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:56.373386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940daaf3-3b09-41ad-8df9-0638fd61b643 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:56.373664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940daaf3-3b09-41ad-8df9-0638fd61b643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:56.380175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940daaf3-3b09-41ad-8df9-0638fd61b643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:56.381629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940daaf3-3b09-41ad-8df9-0638fd61b643 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:56.388459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-940daaf3-3b09-41ad-8df9-0638fd61b643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:50:56.396077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:56.400545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:56.401621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:50:56.401621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:50:56.401621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:50:56.401621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:50:56.401621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:50:56.401621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:50:56.401621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:50:56.401621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:56.401621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 258/513] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:56.401621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940daaf3-3b09-41ad-8df9-0638fd61b643 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.448180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 256/514] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:56.449023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d02ea5-001a-482d-b3ae-6377f0cde498 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.502216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940daaf3-3b09-41ad-8df9-0638fd61b643 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:56.509915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:56.536929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadbbe2-b0a7-4068-b041-e0db9e132a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:56.537202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadbbe2-b0a7-4068-b041-e0db9e132a5b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:56.537688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadbbe2-b0a7-4068-b041-e0db9e132a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:56.537813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadbbe2-b0a7-4068-b041-e0db9e132a5b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:56.545779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadbbe2-b0a7-4068-b041-e0db9e132a5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:56.547234 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadbbe2-b0a7-4068-b041-e0db9e132a5b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:56.548053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d02ea5-001a-482d-b3ae-6377f0cde498 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-feadbbe2-b0a7-4068-b041-e0db9e132a5b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:50:56.563108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:56.564068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:56.564972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 259/515] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:50:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:56.565571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadbbe2-b0a7-4068-b041-e0db9e132a5b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.587232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c037fe3-1ca0-4ef2-af95-b8c1c27193f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:56.587903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c037fe3-1ca0-4ef2-af95-b8c1c27193f1 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:56.588186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c037fe3-1ca0-4ef2-af95-b8c1c27193f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:56.588470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c037fe3-1ca0-4ef2-af95-b8c1c27193f1 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:56.589198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c037fe3-1ca0-4ef2-af95-b8c1c27193f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:56.590637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c037fe3-1ca0-4ef2-af95-b8c1c27193f1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:56.660965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadbbe2-b0a7-4068-b041-e0db9e132a5b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:56.693826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb30b3-6975-440d-b7e1-a832668f67f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:56.694484 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb30b3-6975-440d-b7e1-a832668f67f3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:56.694772 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb30b3-6975-440d-b7e1-a832668f67f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:56.695052 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb30b3-6975-440d-b7e1-a832668f67f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:56.695808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb30b3-6975-440d-b7e1-a832668f67f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:56.696120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb30b3-6975-440d-b7e1-a832668f67f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:56.697502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb30b3-6975-440d-b7e1-a832668f67f3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:56.705244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 260/516] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:50:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:56.706023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb30b3-6975-440d-b7e1-a832668f67f3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.725078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:56.725078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 257/517] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:50:56 2022] POST /identity/v3/users => generated 247 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:50:56.725078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c037fe3-1ca0-4ef2-af95-b8c1c27193f1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.741933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:56.822244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb30b3-6975-440d-b7e1-a832668f67f3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:56.825095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c037fe3-1ca0-4ef2-af95-b8c1c27193f1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:56.860755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16225702-99d7-4565-af31-30a7033f011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:56.861501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16225702-99d7-4565-af31-30a7033f011d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:56.861730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16225702-99d7-4565-af31-30a7033f011d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:56.862005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16225702-99d7-4565-af31-30a7033f011d None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:56.862917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16225702-99d7-4565-af31-30a7033f011d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:56.863224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16225702-99d7-4565-af31-30a7033f011d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:56.865187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16225702-99d7-4565-af31-30a7033f011d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:56.865293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0277c-57a2-4fdf-b859-1be74da88aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:56.865693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0277c-57a2-4fdf-b859-1be74da88aff None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:56.865978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0277c-57a2-4fdf-b859-1be74da88aff None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:56.866801 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0277c-57a2-4fdf-b859-1be74da88aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:56.880578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0277c-57a2-4fdf-b859-1be74da88aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:56.882109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0277c-57a2-4fdf-b859-1be74da88aff None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:56.886867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 261/518] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:50:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:56.887735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16225702-99d7-4565-af31-30a7033f011d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-36a0277c-57a2-4fdf-b859-1be74da88aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:56.901432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:56.902325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:56.903177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:56.903177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:56.903177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:56.903177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:56.903177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 258/519] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:56.903833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0277c-57a2-4fdf-b859-1be74da88aff None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:56.963234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:56.989115 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:57.002757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16225702-99d7-4565-af31-30a7033f011d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.012636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0277c-57a2-4fdf-b859-1be74da88aff None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.044243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd405f-e820-4099-a288-d256136275bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:57.044943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd405f-e820-4099-a288-d256136275bb None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:57.045226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd405f-e820-4099-a288-d256136275bb None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:57.045506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd405f-e820-4099-a288-d256136275bb None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:57.046319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd405f-e820-4099-a288-d256136275bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:57.046625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd405f-e820-4099-a288-d256136275bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:57.048042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd405f-e820-4099-a288-d256136275bb None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:57.049384 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cb555-e1e5-45c1-9623-a43a5f04e3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:57.050202 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cb555-e1e5-45c1-9623-a43a5f04e3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:57.050486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cb555-e1e5-45c1-9623-a43a5f04e3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:57.050761 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cb555-e1e5-45c1-9623-a43a5f04e3b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:57.061992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347cb555-e1e5-45c1-9623-a43a5f04e3b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:57.063454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347cb555-e1e5-45c1-9623-a43a5f04e3b7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:57.065965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 262/520] 10.222.0.45 () {56 vars in 1137 bytes} [Tue Jul 5 15:50:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:57.071244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd405f-e820-4099-a288-d256136275bb None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-347cb555-e1e5-45c1-9623-a43a5f04e3b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:57.081824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:57.082667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:57.083501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:57.083501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:57.083501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:57.083501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:57.083501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 259/521] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:57.083892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cb555-e1e5-45c1-9623-a43a5f04e3b7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:57.178816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd405f-e820-4099-a288-d256136275bb None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.192798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cb555-e1e5-45c1-9623-a43a5f04e3b7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.195178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:57.215498 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b307bc4-8fd3-4667-890f-6821b10277e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:57.216147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b307bc4-8fd3-4667-890f-6821b10277e0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:57.216428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b307bc4-8fd3-4667-890f-6821b10277e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:57.216746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b307bc4-8fd3-4667-890f-6821b10277e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:57.217605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b307bc4-8fd3-4667-890f-6821b10277e0 None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=ef95ab7b924d456195a8761e894d911b) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:57.218099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b307bc4-8fd3-4667-890f-6821b10277e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:57.219395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b307bc4-8fd3-4667-890f-6821b10277e0 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:57.229698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afe875-b7b1-484f-b3f1-18c7215f85d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:57.230377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afe875-b7b1-484f-b3f1-18c7215f85d7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:57.230666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afe875-b7b1-484f-b3f1-18c7215f85d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:57.230955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afe875-b7b1-484f-b3f1-18c7215f85d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:57.231702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afe875-b7b1-484f-b3f1-18c7215f85d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:57.232017 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afe875-b7b1-484f-b3f1-18c7215f85d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:57.233214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 263/522] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:57 2022] GET /identity/v3/role_assignments?user.id=ef95ab7b924d456195a8761e894d911b&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:57.233477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afe875-b7b1-484f-b3f1-18c7215f85d7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:57.234597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b307bc4-8fd3-4667-890f-6821b10277e0 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:57.246594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 260/523] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:57.247485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afe875-b7b1-484f-b3f1-18c7215f85d7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:57.249830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:57.357126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b307bc4-8fd3-4667-890f-6821b10277e0 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.382185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afe875-b7b1-484f-b3f1-18c7215f85d7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.394963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554cd2e-cf31-4e2a-a2d1-53fff74e3269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:57.395689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554cd2e-cf31-4e2a-a2d1-53fff74e3269 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:57.396004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554cd2e-cf31-4e2a-a2d1-53fff74e3269 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:57.396410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554cd2e-cf31-4e2a-a2d1-53fff74e3269 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:57.421896 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0009-26fc-478d-af9f-61def48db1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:57.423252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0009-26fc-478d-af9f-61def48db1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:57.423731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0009-26fc-478d-af9f-61def48db1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:57.424291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af0009-26fc-478d-af9f-61def48db1c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:57.425385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af0009-26fc-478d-af9f-61def48db1c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:57.425887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af0009-26fc-478d-af9f-61def48db1c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:57.427779 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af0009-26fc-478d-af9f-61def48db1c3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:57.441397 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 261/524] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:57.446000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0009-26fc-478d-af9f-61def48db1c3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:57.473113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:57.473407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554cd2e-cf31-4e2a-a2d1-53fff74e3269 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:57.475397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554cd2e-cf31-4e2a-a2d1-53fff74e3269 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:57.489828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:57.559048 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af0009-26fc-478d-af9f-61def48db1c3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.596314 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25f42b-10ab-44ea-abbf-43090f2b6e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:57.597518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25f42b-10ab-44ea-abbf-43090f2b6e93 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:57.598015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25f42b-10ab-44ea-abbf-43090f2b6e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:57.598483 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25f42b-10ab-44ea-abbf-43090f2b6e93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:57.606160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25f42b-10ab-44ea-abbf-43090f2b6e93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:57.607917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25f42b-10ab-44ea-abbf-43090f2b6e93 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:57.618232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 264/525] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:57.618232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554cd2e-cf31-4e2a-a2d1-53fff74e3269 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2e25f42b-10ab-44ea-abbf-43090f2b6e93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:50:57.629160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:57.630006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:57.630815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:50:57.630815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:50:57.630815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:50:57.630815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:50:57.630815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:50:57.630815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:50:57.630815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:50:57.630815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:57.632937 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 262/526] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:57.634139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25f42b-10ab-44ea-abbf-43090f2b6e93 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:57.728794 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:57.736026 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554cd2e-cf31-4e2a-a2d1-53fff74e3269 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.749069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:57.751138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25f42b-10ab-44ea-abbf-43090f2b6e93 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.779986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4efb7-8f08-4c23-b420-d7b0905166a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:57.780828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4efb7-8f08-4c23-b420-d7b0905166a9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:57.781147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4efb7-8f08-4c23-b420-d7b0905166a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:57.781825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4efb7-8f08-4c23-b420-d7b0905166a9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:57.791530 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4efb7-8f08-4c23-b420-d7b0905166a9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:57.793094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4efb7-8f08-4c23-b420-d7b0905166a9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:57.793581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36160b-0db0-4dfe-bbad-bfa18b4b5a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:57.794658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36160b-0db0-4dfe-bbad-bfa18b4b5a45 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:57.795147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36160b-0db0-4dfe-bbad-bfa18b4b5a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:57.795608 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36160b-0db0-4dfe-bbad-bfa18b4b5a45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:57.807405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36160b-0db0-4dfe-bbad-bfa18b4b5a45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:57.809198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36160b-0db0-4dfe-bbad-bfa18b4b5a45 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-7dc4efb7-8f08-4c23-b420-d7b0905166a9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:50:57.810820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:57.811657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 265/527] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:50:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:57.818073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4efb7-8f08-4c23-b420-d7b0905166a9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-de36160b-0db0-4dfe-bbad-bfa18b4b5a45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:57.838489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:57.839386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:57.841199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:57.841199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:57.841199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:57.841199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:57.845539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 263/528] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:57.845539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36160b-0db0-4dfe-bbad-bfa18b4b5a45 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:57.935501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4efb7-8f08-4c23-b420-d7b0905166a9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.973286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621ac5e-b254-4236-b7ce-f1ca373e5922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:57.973952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621ac5e-b254-4236-b7ce-f1ca373e5922 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:57.974237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621ac5e-b254-4236-b7ce-f1ca373e5922 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:57.974605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621ac5e-b254-4236-b7ce-f1ca373e5922 None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:57.975508 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621ac5e-b254-4236-b7ce-f1ca373e5922 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:57.975823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621ac5e-b254-4236-b7ce-f1ca373e5922 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:57.977547 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621ac5e-b254-4236-b7ce-f1ca373e5922 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:57.982262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36160b-0db0-4dfe-bbad-bfa18b4b5a45 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:57.991139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 266/529] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:50:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:57.991861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621ac5e-b254-4236-b7ce-f1ca373e5922 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:58.024375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:58.027942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b104e7-ccbc-45f4-bcb3-fd71856228fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:58.028807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b104e7-ccbc-45f4-bcb3-fd71856228fa None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:58.029102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b104e7-ccbc-45f4-bcb3-fd71856228fa None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:58.030157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b104e7-ccbc-45f4-bcb3-fd71856228fa None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:58.031034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b104e7-ccbc-45f4-bcb3-fd71856228fa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:58.031355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b104e7-ccbc-45f4-bcb3-fd71856228fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:58.032830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b104e7-ccbc-45f4-bcb3-fd71856228fa None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:58.036577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:58.057664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 264/530] 10.222.0.45 () {56 vars in 1139 bytes} [Tue Jul 5 15:50:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:58.058923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b104e7-ccbc-45f4-bcb3-fd71856228fa None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:58.103351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621ac5e-b254-4236-b7ce-f1ca373e5922 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:58.142510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb13a3-46d6-4cbe-89b5-7ba790c56ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:58.143225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb13a3-46d6-4cbe-89b5-7ba790c56ccc None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:58.143530 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb13a3-46d6-4cbe-89b5-7ba790c56ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:58.143863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb13a3-46d6-4cbe-89b5-7ba790c56ccc None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:58.144598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb13a3-46d6-4cbe-89b5-7ba790c56ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:58.145035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb13a3-46d6-4cbe-89b5-7ba790c56ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:58.146502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb13a3-46d6-4cbe-89b5-7ba790c56ccc None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:58.158737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 267/531] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:58.160176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb13a3-46d6-4cbe-89b5-7ba790c56ccc None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:58.193926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b104e7-ccbc-45f4-bcb3-fd71856228fa None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:58.243126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d8831-a35f-4a8f-8c7c-783615e73bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:58.249156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d8831-a35f-4a8f-8c7c-783615e73bcf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:58.249796 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d8831-a35f-4a8f-8c7c-783615e73bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:58.250285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d8831-a35f-4a8f-8c7c-783615e73bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:58.251838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d8831-a35f-4a8f-8c7c-783615e73bcf None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=805a9dd2d0934359b7ca5fe629b96d38) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:58.253302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d8831-a35f-4a8f-8c7c-783615e73bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:58.255675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d8831-a35f-4a8f-8c7c-783615e73bcf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:58.265378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 265/532] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:58 2022] GET /identity/v3/role_assignments?user.id=805a9dd2d0934359b7ca5fe629b96d38&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:58.266440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d8831-a35f-4a8f-8c7c-783615e73bcf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:58.267067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:58.278211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb13a3-46d6-4cbe-89b5-7ba790c56ccc None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:58.318379 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f4e9b-d24f-40e8-93f9-86b19b70957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:58.319306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f4e9b-d24f-40e8-93f9-86b19b70957c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:58.319691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f4e9b-d24f-40e8-93f9-86b19b70957c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:58.320066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f4e9b-d24f-40e8-93f9-86b19b70957c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:58.352345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:58.425783 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d8831-a35f-4a8f-8c7c-783615e73bcf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:58.444156 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f4e9b-d24f-40e8-93f9-86b19b70957c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:58.445594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f4e9b-d24f-40e8-93f9-86b19b70957c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:58.537602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4d490-bdd2-4a62-a9ea-5f05d5a52dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:58.538465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4d490-bdd2-4a62-a9ea-5f05d5a52dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:58.538748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4d490-bdd2-4a62-a9ea-5f05d5a52dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:58.539021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4d490-bdd2-4a62-a9ea-5f05d5a52dc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:58.553439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4d490-bdd2-4a62-a9ea-5f05d5a52dc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:58.555095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4d490-bdd2-4a62-a9ea-5f05d5a52dc5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1ba4d490-bdd2-4a62-a9ea-5f05d5a52dc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:50:58.579952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:58.580893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:58.582239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:50:58.582239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:50:58.582239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:50:58.582239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:50:58.582239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:50:58.582239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:50:58.582239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:50:58.582239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:58.583076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 266/533] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:58.589277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4d490-bdd2-4a62-a9ea-5f05d5a52dc5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:58.592380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:58.613141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 268/534] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:50:58.619467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f4e9b-d24f-40e8-93f9-86b19b70957c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:58.772891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4d490-bdd2-4a62-a9ea-5f05d5a52dc5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:58.785934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f4e9b-d24f-40e8-93f9-86b19b70957c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:58.810939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:58.821892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b51c41-33f3-4e86-b73d-3e21b822b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:58.822705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b51c41-33f3-4e86-b73d-3e21b822b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:58.822986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b51c41-33f3-4e86-b73d-3e21b822b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:58.823263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b51c41-33f3-4e86-b73d-3e21b822b2b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:58.834392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b51c41-33f3-4e86-b73d-3e21b822b2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:58.835844 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b51c41-33f3-4e86-b73d-3e21b822b2b1 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:58.850264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271370c-2336-47aa-8c02-3b806abb2c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:58.850915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271370c-2336-47aa-8c02-3b806abb2c39 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:58.851212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271370c-2336-47aa-8c02-3b806abb2c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:58.859494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271370c-2336-47aa-8c02-3b806abb2c39 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:58.859494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271370c-2336-47aa-8c02-3b806abb2c39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:58.859494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271370c-2336-47aa-8c02-3b806abb2c39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:58.859494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271370c-2336-47aa-8c02-3b806abb2c39 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-29b51c41-33f3-4e86-b73d-3e21b822b2b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:50:58.859764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:58.860598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:58.861480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:50:58.861480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:50:58.861480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:50:58.861480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:58.861480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 269/535] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:58.861480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b51c41-33f3-4e86-b73d-3e21b822b2b1 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:58.889454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 267/536] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:50:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:58.895784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271370c-2336-47aa-8c02-3b806abb2c39 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:58.909158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:58.965216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b51c41-33f3-4e86-b73d-3e21b822b2b1 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:59.008599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271370c-2336-47aa-8c02-3b806abb2c39 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:59.010256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500c7c2-ca3f-4975-a27f-2a3d05fccddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:59.012223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500c7c2-ca3f-4975-a27f-2a3d05fccddc None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.012699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500c7c2-ca3f-4975-a27f-2a3d05fccddc None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.013168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500c7c2-ca3f-4975-a27f-2a3d05fccddc None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.014047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500c7c2-ca3f-4975-a27f-2a3d05fccddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:59.014503 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500c7c2-ca3f-4975-a27f-2a3d05fccddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:59.016020 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500c7c2-ca3f-4975-a27f-2a3d05fccddc None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:59.033061 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 270/537] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:59.034535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdd78-e685-491e-acad-2ff8f5c7a126 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.034954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdd78-e685-491e-acad-2ff8f5c7a126 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.035359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdd78-e685-491e-acad-2ff8f5c7a126 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.040846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 271/538] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:59.041636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cdd78-e685-491e-acad-2ff8f5c7a126 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:59.056797 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8983861-56b3-46b8-be2a-22be2d96c762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:59.057455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8983861-56b3-46b8-be2a-22be2d96c762 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.057739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8983861-56b3-46b8-be2a-22be2d96c762 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.058013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8983861-56b3-46b8-be2a-22be2d96c762 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.058844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8983861-56b3-46b8-be2a-22be2d96c762 None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=a5b712bd7d8d4930a3bfc99581be8cb7) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:59.059151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8983861-56b3-46b8-be2a-22be2d96c762 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:59.060647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8983861-56b3-46b8-be2a-22be2d96c762 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:59.067537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 268/539] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:58 2022] GET /identity/v3/role_assignments?user.id=a5b712bd7d8d4930a3bfc99581be8cb7&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:59.068921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5edda7-0e19-4ef8-bc2a-e1da97e1f5c5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.069211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5edda7-0e19-4ef8-bc2a-e1da97e1f5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.069488 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5edda7-0e19-4ef8-bc2a-e1da97e1f5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.087329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3e5edda7-0e19-4ef8-bc2a-e1da97e1f5c5 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:59.088576 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3e5edda7-0e19-4ef8-bc2a-e1da97e1f5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:59.133688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:59.198952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 269/540] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:59.219040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cdd78-e685-491e-acad-2ff8f5c7a126 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:59.221082 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:59.240532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5edda7-0e19-4ef8-bc2a-e1da97e1f5c5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:59.264562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce368e2-40bc-483f-ba4a-9644af7ed9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:59.265849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce368e2-40bc-483f-ba4a-9644af7ed9ac None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.266143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce368e2-40bc-483f-ba4a-9644af7ed9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.266424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce368e2-40bc-483f-ba4a-9644af7ed9ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.280811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce368e2-40bc-483f-ba4a-9644af7ed9ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:59.280811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce368e2-40bc-483f-ba4a-9644af7ed9ac None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-5ce368e2-40bc-483f-ba4a-9644af7ed9ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:50:59.305505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:59.306343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:59.307164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:50:59.307164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:50:59.307164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:50:59.307164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:50:59.307164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:50:59.307164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:50:59.307164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:50:59.307164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:50:59.313712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 272/541] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:50:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:50:59.314597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce368e2-40bc-483f-ba4a-9644af7ed9ac None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:59.440951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5edda7-0e19-4ef8-bc2a-e1da97e1f5c5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:59.506471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557c252-b083-426a-9799-6bbf1dce622e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:59.507267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557c252-b083-426a-9799-6bbf1dce622e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.507548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557c252-b083-426a-9799-6bbf1dce622e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.507822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557c252-b083-426a-9799-6bbf1dce622e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.529668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce368e2-40bc-483f-ba4a-9644af7ed9ac None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:59.532119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557c252-b083-426a-9799-6bbf1dce622e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c557c252-b083-426a-9799-6bbf1dce622e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:50:59.541574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:50:59.542522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:50:59.543300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:50:59.543897 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 270/542] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:50:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:50:59.552370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557c252-b083-426a-9799-6bbf1dce622e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:59.596159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965d148-ebe9-4c50-b513-11095bb4369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:59.597905 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965d148-ebe9-4c50-b513-11095bb4369f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.598838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965d148-ebe9-4c50-b513-11095bb4369f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.599323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965d148-ebe9-4c50-b513-11095bb4369f None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.600324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965d148-ebe9-4c50-b513-11095bb4369f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:59.600873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965d148-ebe9-4c50-b513-11095bb4369f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:59.603332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965d148-ebe9-4c50-b513-11095bb4369f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:59.616711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:59.618289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 273/543] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:59.619884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d3679-5860-4e80-b804-25657cf6a825 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.620825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d3679-5860-4e80-b804-25657cf6a825 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.621324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d3679-5860-4e80-b804-25657cf6a825 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.623253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 274/544] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:50:59.624174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d3679-5860-4e80-b804-25657cf6a825 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:59.649117 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:50:59.692734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557c252-b083-426a-9799-6bbf1dce622e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:59.739725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeae163-d245-4ed8-b6b0-c515df50a4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:59.741242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeae163-d245-4ed8-b6b0-c515df50a4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.741699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeae163-d245-4ed8-b6b0-c515df50a4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.742112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeae163-d245-4ed8-b6b0-c515df50a4f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.743638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeae163-d245-4ed8-b6b0-c515df50a4f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:59.744378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeae163-d245-4ed8-b6b0-c515df50a4f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:59.746055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeae163-d245-4ed8-b6b0-c515df50a4f2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:59.747023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d3679-5860-4e80-b804-25657cf6a825 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:50:59.756553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 271/545] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:50:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:59.758005 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d9adb-b64b-4d29-aa29-d929fed7aa71 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.758418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d9adb-b64b-4d29-aa29-d929fed7aa71 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.758816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d9adb-b64b-4d29-aa29-d929fed7aa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.778612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-451d9adb-b64b-4d29-aa29-d929fed7aa71 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:50:59.779887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-451d9adb-b64b-4d29-aa29-d929fed7aa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:50:59.794485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ec074-e966-4ead-8f87-0f9b45db741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:50:59.795330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec074-e966-4ead-8f87-0f9b45db741e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:50:59.795824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec074-e966-4ead-8f87-0f9b45db741e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:50:59.796291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec074-e966-4ead-8f87-0f9b45db741e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:50:59.797368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ec074-e966-4ead-8f87-0f9b45db741e None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=887d2ad3cbf7459a8a59e7442ba9744e) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:50:59.797853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ec074-e966-4ead-8f87-0f9b45db741e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:50:59.799508 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ec074-e966-4ead-8f87-0f9b45db741e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:50:59.806070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 275/546] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:50:59 2022] GET /identity/v3/role_assignments?user.id=887d2ad3cbf7459a8a59e7442ba9744e&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:50:59.807150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ec074-e966-4ead-8f87-0f9b45db741e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:59.877670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 272/547] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:50:59.900549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d9adb-b64b-4d29-aa29-d929fed7aa71 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:50:59.964327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:00.023298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:00.048737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ec074-e966-4ead-8f87-0f9b45db741e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:00.119063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d9adb-b64b-4d29-aa29-d929fed7aa71 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:00.119804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6116120-5269-46ff-b304-758435171bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:00.120604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6116120-5269-46ff-b304-758435171bde None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:00.120916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6116120-5269-46ff-b304-758435171bde None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:00.121195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6116120-5269-46ff-b304-758435171bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:00.142558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6116120-5269-46ff-b304-758435171bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:00.144597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6116120-5269-46ff-b304-758435171bde None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-c6116120-5269-46ff-b304-758435171bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:00.163113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:00.164075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:00.164912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:00.164912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:00.164912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:00.164912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:00.164912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 276/548] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:00.170261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6116120-5269-46ff-b304-758435171bde None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:00.178738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969f56a-32f2-4c09-b25e-fa749cdeca9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:00.179619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969f56a-32f2-4c09-b25e-fa749cdeca9c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:00.179971 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969f56a-32f2-4c09-b25e-fa749cdeca9c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:00.180299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969f56a-32f2-4c09-b25e-fa749cdeca9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:00.206688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969f56a-32f2-4c09-b25e-fa749cdeca9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e969f56a-32f2-4c09-b25e-fa749cdeca9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:00.215634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:00.217902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:00.219834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 273/549] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:50:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:00.227869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969f56a-32f2-4c09-b25e-fa749cdeca9c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:00.340236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6116120-5269-46ff-b304-758435171bde None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:00.387087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:00.406284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969f56a-32f2-4c09-b25e-fa749cdeca9c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:00.435383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa91175-4361-4a0e-9b35-5ac1b4c0da2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:00.436413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa91175-4361-4a0e-9b35-5ac1b4c0da2b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:00.436925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa91175-4361-4a0e-9b35-5ac1b4c0da2b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:00.437393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa91175-4361-4a0e-9b35-5ac1b4c0da2b None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:00.438370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa91175-4361-4a0e-9b35-5ac1b4c0da2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:00.439553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa91175-4361-4a0e-9b35-5ac1b4c0da2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:00.444106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa91175-4361-4a0e-9b35-5ac1b4c0da2b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:00.453104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:00.465171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 277/550] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:00.470346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa91175-4361-4a0e-9b35-5ac1b4c0da2b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:00.477300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dd96c-3d94-46f7-a776-48b793d22201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:00.478589 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dd96c-3d94-46f7-a776-48b793d22201 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:00.479194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dd96c-3d94-46f7-a776-48b793d22201 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:00.480150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dd96c-3d94-46f7-a776-48b793d22201 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:00.483621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dd96c-3d94-46f7-a776-48b793d22201 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:00.484136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dd96c-3d94-46f7-a776-48b793d22201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:00.487324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dd96c-3d94-46f7-a776-48b793d22201 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:00.498116 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 274/551] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:00.500351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be364473-f4c2-4a76-ac49-7052278eaac8 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:00.500883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be364473-f4c2-4a76-ac49-7052278eaac8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:00.501450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be364473-f4c2-4a76-ac49-7052278eaac8 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:00.503361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 275/552] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:00.504320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be364473-f4c2-4a76-ac49-7052278eaac8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:00.610001 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa91175-4361-4a0e-9b35-5ac1b4c0da2b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:00.618842 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be364473-f4c2-4a76-ac49-7052278eaac8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:00.647228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbe00b-6dcd-4d29-b576-d7f186888a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:00.647955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbe00b-6dcd-4d29-b576-d7f186888a11 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:00.648242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbe00b-6dcd-4d29-b576-d7f186888a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:00.648545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbe00b-6dcd-4d29-b576-d7f186888a11 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:00.656726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbe00b-6dcd-4d29-b576-d7f186888a11 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:00.657249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42355cd0-17cf-4246-824c-4cc4443349fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:00.658236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbe00b-6dcd-4d29-b576-d7f186888a11 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:00.658534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42355cd0-17cf-4246-824c-4cc4443349fb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:00.659057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42355cd0-17cf-4246-824c-4cc4443349fb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:00.660108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42355cd0-17cf-4246-824c-4cc4443349fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:00.671782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42355cd0-17cf-4246-824c-4cc4443349fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:00.673604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42355cd0-17cf-4246-824c-4cc4443349fb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-8acbe00b-6dcd-4d29-b576-d7f186888a11 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:00.679605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:00.680532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:00.682806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:00.682806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:00.682806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:00.682806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:00.682806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:00.682806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:00.682806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:00.682806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:00.682806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:00.682806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:00.685705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 278/553] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:00.687882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:00.688720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e25e-e08d-410d-9c39-ca80335f162e None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:00.689215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e25e-e08d-410d-9c39-ca80335f162e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:00.689675 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e25e-e08d-410d-9c39-ca80335f162e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-42355cd0-17cf-4246-824c-4cc4443349fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:00.701283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:00.707326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:00.708232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:00.708232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:00.708232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:00.708232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:00.708232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 276/554] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:00.708232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42355cd0-17cf-4246-824c-4cc4443349fb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:00.711114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-ebf5e25e-e08d-410d-9c39-ca80335f162e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:00.712539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ebf5e25e-e08d-410d-9c39-ca80335f162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:00.775268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:00.793477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 279/555] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:00.794311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e25e-e08d-410d-9c39-ca80335f162e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:00.815222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42355cd0-17cf-4246-824c-4cc4443349fb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:00.854882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142868e1-30d7-4288-be86-2d5f0130cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:00.855898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142868e1-30d7-4288-be86-2d5f0130cf95 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:00.856381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142868e1-30d7-4288-be86-2d5f0130cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:00.856905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142868e1-30d7-4288-be86-2d5f0130cf95 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:00.858118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142868e1-30d7-4288-be86-2d5f0130cf95 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:00.858709 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142868e1-30d7-4288-be86-2d5f0130cf95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:00.860437 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142868e1-30d7-4288-be86-2d5f0130cf95 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:00.880002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 277/556] 10.222.0.45 () {56 vars in 1135 bytes} [Tue Jul 5 15:51:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:00.881270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142868e1-30d7-4288-be86-2d5f0130cf95 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:00.910443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e25e-e08d-410d-9c39-ca80335f162e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:00.941607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:00.948891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86f1b2-fc7e-4848-9ba7-97ca9c484db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:00.949621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86f1b2-fc7e-4848-9ba7-97ca9c484db6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:00.949907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86f1b2-fc7e-4848-9ba7-97ca9c484db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:00.950186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86f1b2-fc7e-4848-9ba7-97ca9c484db6 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:00.950923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86f1b2-fc7e-4848-9ba7-97ca9c484db6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:00.951231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86f1b2-fc7e-4848-9ba7-97ca9c484db6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:00.952735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86f1b2-fc7e-4848-9ba7-97ca9c484db6 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:00.965701 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 280/557] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:00.966545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86f1b2-fc7e-4848-9ba7-97ca9c484db6 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:00.991206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142868e1-30d7-4288-be86-2d5f0130cf95 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:01.022330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:01.029768 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436da95-21a5-463c-9b73-6f0f47e4a4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:01.030898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436da95-21a5-463c-9b73-6f0f47e4a4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.038170 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436da95-21a5-463c-9b73-6f0f47e4a4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.038170 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436da95-21a5-463c-9b73-6f0f47e4a4b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.042934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436da95-21a5-463c-9b73-6f0f47e4a4b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a436da95-21a5-463c-9b73-6f0f47e4a4b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:01.051177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:01.052584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:01.053770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:01.056295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 278/558] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:01.057434 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436da95-21a5-463c-9b73-6f0f47e4a4b1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:01.077789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86f1b2-fc7e-4848-9ba7-97ca9c484db6 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:01.120608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666e1ac-ffec-43cd-9985-f4b9c17d48bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:01.121301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666e1ac-ffec-43cd-9985-f4b9c17d48bf None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.121589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666e1ac-ffec-43cd-9985-f4b9c17d48bf None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.121868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666e1ac-ffec-43cd-9985-f4b9c17d48bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.177367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436da95-21a5-463c-9b73-6f0f47e4a4b1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:01.195538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:01.200563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666e1ac-ffec-43cd-9985-f4b9c17d48bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:01.202051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666e1ac-ffec-43cd-9985-f4b9c17d48bf None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:01.215980 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ccff5-b106-472d-ae67-dddfaad02e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:01.217239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ccff5-b106-472d-ae67-dddfaad02e82 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.217917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ccff5-b106-472d-ae67-dddfaad02e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.218392 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ccff5-b106-472d-ae67-dddfaad02e82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.227588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ccff5-b106-472d-ae67-dddfaad02e82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:01.229615 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ccff5-b106-472d-ae67-dddfaad02e82 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-975ccff5-b106-472d-ae67-dddfaad02e82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:01.247277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:01.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:01.249266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:01.249266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:01.249266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:01.249266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:01.249266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:01.249266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:01.249266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:01.249266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:01.249266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:01.249266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:01.252046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 279/559] 10.222.0.45 () {56 vars in 1093 bytes} [Tue Jul 5 15:51:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:01.253748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ccff5-b106-472d-ae67-dddfaad02e82 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:01.254537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:01.326584 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 281/560] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:51:01.327688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666e1ac-ffec-43cd-9985-f4b9c17d48bf None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:01.363027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ccff5-b106-472d-ae67-dddfaad02e82 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:01.398685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e3951-42ea-4d09-b648-85c747128a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:01.399451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e3951-42ea-4d09-b648-85c747128a09 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.399737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e3951-42ea-4d09-b648-85c747128a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.400008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e3951-42ea-4d09-b648-85c747128a09 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.400748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e3951-42ea-4d09-b648-85c747128a09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:01.401076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e3951-42ea-4d09-b648-85c747128a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:01.402470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e3951-42ea-4d09-b648-85c747128a09 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:01.411678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 280/561] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:01.412465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e3951-42ea-4d09-b648-85c747128a09 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:01.449031 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666e1ac-ffec-43cd-9985-f4b9c17d48bf None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:01.473151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:01.494116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b73e81-a409-4515-8691-cbe2d7d36187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:01.494870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b73e81-a409-4515-8691-cbe2d7d36187 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.495294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b73e81-a409-4515-8691-cbe2d7d36187 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.495852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b73e81-a409-4515-8691-cbe2d7d36187 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.496897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b73e81-a409-4515-8691-cbe2d7d36187 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:01.497208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b73e81-a409-4515-8691-cbe2d7d36187 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:01.498678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b73e81-a409-4515-8691-cbe2d7d36187 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:01.521974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 282/562] 10.222.0.45 () {56 vars in 1133 bytes} [Tue Jul 5 15:51:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:01.523114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2495d-3071-43d0-a9e0-0156e68fc20d None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.523408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2495d-3071-43d0-a9e0-0156e68fc20d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.523689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2495d-3071-43d0-a9e0-0156e68fc20d None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.529057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 283/563] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:01.529809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2495d-3071-43d0-a9e0-0156e68fc20d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:01.538302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:01.539842 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e3951-42ea-4d09-b648-85c747128a09 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:01.577115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8d007-b343-4ce0-8eed-76ba82bd88ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:01.578000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8d007-b343-4ce0-8eed-76ba82bd88ad None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.578290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8d007-b343-4ce0-8eed-76ba82bd88ad None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.578566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8d007-b343-4ce0-8eed-76ba82bd88ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.589905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8d007-b343-4ce0-8eed-76ba82bd88ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:01.591511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8d007-b343-4ce0-8eed-76ba82bd88ad None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-01b8d007-b343-4ce0-8eed-76ba82bd88ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:01.609338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:01.610351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:01.611301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:01.611301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:01.611301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:01.611301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:01.611301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 281/564] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:01.611578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8d007-b343-4ce0-8eed-76ba82bd88ad None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:01.640829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2495d-3071-43d0-a9e0-0156e68fc20d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:01.678402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b606f-f4d4-4561-a93a-5d4d8e147af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:01.679177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b606f-f4d4-4561-a93a-5d4d8e147af9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.679450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b606f-f4d4-4561-a93a-5d4d8e147af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.679722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b606f-f4d4-4561-a93a-5d4d8e147af9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.691029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b606f-f4d4-4561-a93a-5d4d8e147af9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:01.692497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b606f-f4d4-4561-a93a-5d4d8e147af9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-130b606f-f4d4-4561-a93a-5d4d8e147af9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:01.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:01.711699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:01.712570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:01.712570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:01.712570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:01.712570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:01.712570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 284/565] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:01.713315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65797b-1c17-490f-8d75-89e74aa8ccfe None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.713497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65797b-1c17-490f-8d75-89e74aa8ccfe None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.713773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65797b-1c17-490f-8d75-89e74aa8ccfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.714116 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:01.726327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8d007-b343-4ce0-8eed-76ba82bd88ad None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:01.731677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1e65797b-1c17-490f-8d75-89e74aa8ccfe None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:01.734913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1e65797b-1c17-490f-8d75-89e74aa8ccfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:01.765235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895a078-66b6-4a11-9964-3ca2d4f3850f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:01.767229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895a078-66b6-4a11-9964-3ca2d4f3850f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.767229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895a078-66b6-4a11-9964-3ca2d4f3850f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.767229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895a078-66b6-4a11-9964-3ca2d4f3850f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.781248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895a078-66b6-4a11-9964-3ca2d4f3850f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:01.783077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895a078-66b6-4a11-9964-3ca2d4f3850f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:01.799666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-d895a078-66b6-4a11-9964-3ca2d4f3850f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:01.805438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:01.806389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:01.807278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:01.807278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:01.807278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:01.807278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:01.807278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 282/566] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:01.808226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895a078-66b6-4a11-9964-3ca2d4f3850f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:01.822589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 285/567] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:01.823690 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65797b-1c17-490f-8d75-89e74aa8ccfe None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:01.956470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65797b-1c17-490f-8d75-89e74aa8ccfe None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:01.962999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895a078-66b6-4a11-9964-3ca2d4f3850f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:01.990764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77d424-f4ad-4429-a727-3892cc0a273c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:01.991425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77d424-f4ad-4429-a727-3892cc0a273c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:01.991706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77d424-f4ad-4429-a727-3892cc0a273c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:01.991977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77d424-f4ad-4429-a727-3892cc0a273c None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:01.992706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77d424-f4ad-4429-a727-3892cc0a273c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:01.993036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77d424-f4ad-4429-a727-3892cc0a273c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:01.994393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77d424-f4ad-4429-a727-3892cc0a273c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.002449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92af0f5-c7e7-4701-9e0a-0596bc034d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.003138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92af0f5-c7e7-4701-9e0a-0596bc034d9c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.003421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92af0f5-c7e7-4701-9e0a-0596bc034d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.003701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92af0f5-c7e7-4701-9e0a-0596bc034d9c None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.004418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92af0f5-c7e7-4701-9e0a-0596bc034d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:02.004725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92af0f5-c7e7-4701-9e0a-0596bc034d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:02.010360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 286/568] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:02.010496 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92af0f5-c7e7-4701-9e0a-0596bc034d9c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.010589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77d424-f4ad-4429-a727-3892cc0a273c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.023927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 283/569] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:02.024945 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92af0f5-c7e7-4701-9e0a-0596bc034d9c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.029036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:02.067280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:02.120587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77d424-f4ad-4429-a727-3892cc0a273c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.124678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92af0f5-c7e7-4701-9e0a-0596bc034d9c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.153320 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327880d-78f2-47ab-8214-e827e30ef40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.153975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327880d-78f2-47ab-8214-e827e30ef40a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.154258 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327880d-78f2-47ab-8214-e827e30ef40a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.154535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327880d-78f2-47ab-8214-e827e30ef40a None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.155251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327880d-78f2-47ab-8214-e827e30ef40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:02.155614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327880d-78f2-47ab-8214-e827e30ef40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:02.157009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327880d-78f2-47ab-8214-e827e30ef40a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.157819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238849e-f237-49ff-bf37-22df5810a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.158601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238849e-f237-49ff-bf37-22df5810a93e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.158881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238849e-f237-49ff-bf37-22df5810a93e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.159716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238849e-f237-49ff-bf37-22df5810a93e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.169019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 287/570] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:02.169803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327880d-78f2-47ab-8214-e827e30ef40a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.170154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238849e-f237-49ff-bf37-22df5810a93e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e238849e-f237-49ff-bf37-22df5810a93e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:02.177981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:02.179110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 284/571] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:02.184648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238849e-f237-49ff-bf37-22df5810a93e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.242522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:02.270104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327880d-78f2-47ab-8214-e827e30ef40a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.280024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238849e-f237-49ff-bf37-22df5810a93e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.280260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:02.305501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92241bc-d71b-4a96-b7ff-0fcd977dd468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.306313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92241bc-d71b-4a96-b7ff-0fcd977dd468 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.306885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92241bc-d71b-4a96-b7ff-0fcd977dd468 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.307363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92241bc-d71b-4a96-b7ff-0fcd977dd468 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.314572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92241bc-d71b-4a96-b7ff-0fcd977dd468 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:02.315907 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363b293-7ffd-4896-a654-423703fa50a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.316030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92241bc-d71b-4a96-b7ff-0fcd977dd468 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.316693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363b293-7ffd-4896-a654-423703fa50a4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.317015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363b293-7ffd-4896-a654-423703fa50a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.317292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363b293-7ffd-4896-a654-423703fa50a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.323884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363b293-7ffd-4896-a654-423703fa50a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:02.325328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363b293-7ffd-4896-a654-423703fa50a4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-f92241bc-d71b-4a96-b7ff-0fcd977dd468 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:02.329912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:02.331545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:02.332515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 288/572] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:02.333488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738885ea-028d-4a54-8300-46ce60baa79e None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.333657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738885ea-028d-4a54-8300-46ce60baa79e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.333933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738885ea-028d-4a54-8300-46ce60baa79e None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.335154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 289/573] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:02.335829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738885ea-028d-4a54-8300-46ce60baa79e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1363b293-7ffd-4896-a654-423703fa50a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:02.344365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:02.345270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:02.346128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:02.346128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:02.346128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:02.346128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:02.346128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:02.346128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:02.346128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:02.346128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:02.347689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 285/574] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:02.348502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363b293-7ffd-4896-a654-423703fa50a4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.437118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738885ea-028d-4a54-8300-46ce60baa79e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.446522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363b293-7ffd-4896-a654-423703fa50a4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.469730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:02.470647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766ea0e-90b8-4c08-b1b0-b637ad22a5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.471392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ea0e-90b8-4c08-b1b0-b637ad22a5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.471704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ea0e-90b8-4c08-b1b0-b637ad22a5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.471981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ea0e-90b8-4c08-b1b0-b637ad22a5f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.479028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766ea0e-90b8-4c08-b1b0-b637ad22a5f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:02.480453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766ea0e-90b8-4c08-b1b0-b637ad22a5f4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.481080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865bab2-aeb8-4356-89fa-cbdfa73211b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.481731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865bab2-aeb8-4356-89fa-cbdfa73211b6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.482013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865bab2-aeb8-4356-89fa-cbdfa73211b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.482291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865bab2-aeb8-4356-89fa-cbdfa73211b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.483024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865bab2-aeb8-4356-89fa-cbdfa73211b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:02.483329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865bab2-aeb8-4356-89fa-cbdfa73211b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:02.484665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865bab2-aeb8-4356-89fa-cbdfa73211b6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.492419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 286/575] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:02.493269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865bab2-aeb8-4356-89fa-cbdfa73211b6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-3766ea0e-90b8-4c08-b1b0-b637ad22a5f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:02.493872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:02.494711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:02.495579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:02.495579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:02.495579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:02.495579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:02.495579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:02.495579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:02.495579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:02.495579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:02.495579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 290/576] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:02.496638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fe602-e240-44bc-b0a5-a71ab3f1c5ae None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.496757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fe602-e240-44bc-b0a5-a71ab3f1c5ae None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.497736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fe602-e240-44bc-b0a5-a71ab3f1c5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.513619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-7e9fe602-e240-44bc-b0a5-a71ab3f1c5ae None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:02.514838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7e9fe602-e240-44bc-b0a5-a71ab3f1c5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:02.537615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:02.579649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 291/577] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:02.580528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fe602-e240-44bc-b0a5-a71ab3f1c5ae None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.606714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865bab2-aeb8-4356-89fa-cbdfa73211b6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.640932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdacc15-47ce-4714-82ae-cd01c393ffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.641588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdacc15-47ce-4714-82ae-cd01c393ffff None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.641873 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdacc15-47ce-4714-82ae-cd01c393ffff None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.642149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdacc15-47ce-4714-82ae-cd01c393ffff None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.642987 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdacc15-47ce-4714-82ae-cd01c393ffff None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:02.643310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdacc15-47ce-4714-82ae-cd01c393ffff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:02.645288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdacc15-47ce-4714-82ae-cd01c393ffff None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.662187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 287/578] 10.222.0.45 () {56 vars in 1137 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:02.662893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdacc15-47ce-4714-82ae-cd01c393ffff None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.681446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fe602-e240-44bc-b0a5-a71ab3f1c5ae None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.703691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:02.715459 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55a421-fb16-4490-bdf5-62ab29c6d208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.716113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55a421-fb16-4490-bdf5-62ab29c6d208 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.716393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55a421-fb16-4490-bdf5-62ab29c6d208 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.716666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55a421-fb16-4490-bdf5-62ab29c6d208 None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.717483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55a421-fb16-4490-bdf5-62ab29c6d208 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:02.717795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55a421-fb16-4490-bdf5-62ab29c6d208 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:02.719222 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55a421-fb16-4490-bdf5-62ab29c6d208 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.730900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 292/579] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:02.731712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55a421-fb16-4490-bdf5-62ab29c6d208 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.755251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:02.763381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdacc15-47ce-4714-82ae-cd01c393ffff None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.796216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243288-16a2-4995-9a7d-f5a34a7382b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.796896 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243288-16a2-4995-9a7d-f5a34a7382b6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.797176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243288-16a2-4995-9a7d-f5a34a7382b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.797450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243288-16a2-4995-9a7d-f5a34a7382b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.798226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63243288-16a2-4995-9a7d-f5a34a7382b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:02.798532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63243288-16a2-4995-9a7d-f5a34a7382b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:02.800494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63243288-16a2-4995-9a7d-f5a34a7382b6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.811625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 288/580] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:02.816638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243288-16a2-4995-9a7d-f5a34a7382b6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.836742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55a421-fb16-4490-bdf5-62ab29c6d208 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.896476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460c30b-2233-42f5-8310-c3671a851031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.897298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460c30b-2233-42f5-8310-c3671a851031 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.897583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460c30b-2233-42f5-8310-c3671a851031 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.897860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460c30b-2233-42f5-8310-c3671a851031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.907795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460c30b-2233-42f5-8310-c3671a851031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:02.909268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460c30b-2233-42f5-8310-c3671a851031 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-3460c30b-2233-42f5-8310-c3671a851031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:02.926908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:02.927967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:02.929030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:02.929030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:02.929030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:02.929030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:02.929030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 293/581] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:02.929030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460c30b-2233-42f5-8310-c3671a851031 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:02.931525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243288-16a2-4995-9a7d-f5a34a7382b6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:02.933629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:02.966455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babb9b9-9a6d-4011-b8ac-d8de3ea7b525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:02.967236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babb9b9-9a6d-4011-b8ac-d8de3ea7b525 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:02.967511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babb9b9-9a6d-4011-b8ac-d8de3ea7b525 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:02.967779 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babb9b9-9a6d-4011-b8ac-d8de3ea7b525 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:02.977884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babb9b9-9a6d-4011-b8ac-d8de3ea7b525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0babb9b9-9a6d-4011-b8ac-d8de3ea7b525 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:02.985441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:02.986556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 289/582] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:02.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babb9b9-9a6d-4011-b8ac-d8de3ea7b525 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:03.001742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:03.033383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460c30b-2233-42f5-8310-c3671a851031 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:03.066098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6440cd-f7c5-487a-ba33-1306fe6c7430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:03.066924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6440cd-f7c5-487a-ba33-1306fe6c7430 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:03.067206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6440cd-f7c5-487a-ba33-1306fe6c7430 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:03.067476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6440cd-f7c5-487a-ba33-1306fe6c7430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:03.078422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6440cd-f7c5-487a-ba33-1306fe6c7430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:03.079872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6440cd-f7c5-487a-ba33-1306fe6c7430 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:03.087852 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babb9b9-9a6d-4011-b8ac-d8de3ea7b525 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-4b6440cd-f7c5-487a-ba33-1306fe6c7430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:03.099716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:03.100714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:03.101938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:03.101938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:03.101938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:03.101938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:03.101938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 294/583] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:03.102219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6440cd-f7c5-487a-ba33-1306fe6c7430 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:03.125629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ed8ee-efef-476c-9bea-f21f3ebc5580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:03.126278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ed8ee-efef-476c-9bea-f21f3ebc5580 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:03.126562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ed8ee-efef-476c-9bea-f21f3ebc5580 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:03.126838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ed8ee-efef-476c-9bea-f21f3ebc5580 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:03.127666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ed8ee-efef-476c-9bea-f21f3ebc5580 None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=8da9cdd70af646928ee0edcadbcb524f) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:03.127978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ed8ee-efef-476c-9bea-f21f3ebc5580 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:03.129437 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ed8ee-efef-476c-9bea-f21f3ebc5580 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:03.134964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 290/584] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:02 2022] GET /identity/v3/role_assignments?user.id=8da9cdd70af646928ee0edcadbcb524f&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:03.135736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ed8ee-efef-476c-9bea-f21f3ebc5580 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:03.166988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:03.224509 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6440cd-f7c5-487a-ba33-1306fe6c7430 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:03.250867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:03.253929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ed8ee-efef-476c-9bea-f21f3ebc5580 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:03.263295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829a9da-1dbb-4075-a60d-108706aff77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:03.264331 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829a9da-1dbb-4075-a60d-108706aff77a None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:03.264618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829a9da-1dbb-4075-a60d-108706aff77a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:03.264932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829a9da-1dbb-4075-a60d-108706aff77a None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/ef95ab7b924d456195a8761e894d911b/roles/ac08a857de8b484fb5dfc65cd4cdc044` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:03.283150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829a9da-1dbb-4075-a60d-108706aff77a None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=ef95ab7b924d456195a8761e894d911b, role_id=ac08a857de8b484fb5dfc65cd4cdc044)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:03.285983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829a9da-1dbb-4075-a60d-108706aff77a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:03.297953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89157339-38d5-43a7-b46c-ea73c330a591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:03.298628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89157339-38d5-43a7-b46c-ea73c330a591 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:03.298915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89157339-38d5-43a7-b46c-ea73c330a591 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:03.299192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89157339-38d5-43a7-b46c-ea73c330a591 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:03.300488 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89157339-38d5-43a7-b46c-ea73c330a591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:03.300820 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89157339-38d5-43a7-b46c-ea73c330a591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:03.302196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89157339-38d5-43a7-b46c-ea73c330a591 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:03.314019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 291/585] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:03.315022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89157339-38d5-43a7-b46c-ea73c330a591 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:03.368801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 295/586] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:51:03 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/ef95ab7b924d456195a8761e894d911b/roles/ac08a857de8b484fb5dfc65cd4cdc044 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:51:03.368801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829a9da-1dbb-4075-a60d-108706aff77a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:03.427408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:03.444598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89157339-38d5-43a7-b46c-ea73c330a591 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:03.485594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025b751-d0e8-4ce7-91b8-5e494a4d6c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:03.486434 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025b751-d0e8-4ce7-91b8-5e494a4d6c5a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:03.486719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025b751-d0e8-4ce7-91b8-5e494a4d6c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:03.487029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025b751-d0e8-4ce7-91b8-5e494a4d6c5a None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:03.487849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025b751-d0e8-4ce7-91b8-5e494a4d6c5a None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:03.488160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025b751-d0e8-4ce7-91b8-5e494a4d6c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:03.489777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025b751-d0e8-4ce7-91b8-5e494a4d6c5a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:03.499533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 292/587] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:03.500394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025b751-d0e8-4ce7-91b8-5e494a4d6c5a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:03.512327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829a9da-1dbb-4075-a60d-108706aff77a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:03.560344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0fa7-5be0-4546-b03a-0b575a0c1dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:03.561116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0fa7-5be0-4546-b03a-0b575a0c1dbe None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:03.561467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0fa7-5be0-4546-b03a-0b575a0c1dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:03.561853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0fa7-5be0-4546-b03a-0b575a0c1dbe None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:03.562639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0fa7-5be0-4546-b03a-0b575a0c1dbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:03.563038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0fa7-5be0-4546-b03a-0b575a0c1dbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:03.564478 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0fa7-5be0-4546-b03a-0b575a0c1dbe None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:03.578219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 296/588] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:03.579037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0fa7-5be0-4546-b03a-0b575a0c1dbe None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:03.621155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:03.623750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025b751-d0e8-4ce7-91b8-5e494a4d6c5a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:03.670540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddc978-66cf-4724-b33c-282565bd54cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:03.671443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddc978-66cf-4724-b33c-282565bd54cd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:03.671793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddc978-66cf-4724-b33c-282565bd54cd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:03.672136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddc978-66cf-4724-b33c-282565bd54cd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:03.683064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddc978-66cf-4724-b33c-282565bd54cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:03.685052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddc978-66cf-4724-b33c-282565bd54cd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:03.701633 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0fa7-5be0-4546-b03a-0b575a0c1dbe None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b4ddc978-66cf-4724-b33c-282565bd54cd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:03.703313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:03.704428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:03.705598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:03.705598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:03.705598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:03.705598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:03.705598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:03.705598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:03.705598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:03.705598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:03.705598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:51:03.705598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:03.710975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 293/589] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:03.710975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddc978-66cf-4724-b33c-282565bd54cd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:03.723553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:03.760381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac06bf-2792-4f68-a722-c79df8e9f041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:03.761495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac06bf-2792-4f68-a722-c79df8e9f041 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:03.762189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac06bf-2792-4f68-a722-c79df8e9f041 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:03.763057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac06bf-2792-4f68-a722-c79df8e9f041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:03.777469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac06bf-2792-4f68-a722-c79df8e9f041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:03.779774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac06bf-2792-4f68-a722-c79df8e9f041 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-76ac06bf-2792-4f68-a722-c79df8e9f041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:03.805668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:03.806741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:03.806741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:03.806741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:03.806741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:03.808568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 297/590] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:03.810003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac06bf-2792-4f68-a722-c79df8e9f041 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:03.824933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddc978-66cf-4724-b33c-282565bd54cd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:03.862930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84126e-429c-4232-bc31-5d47197b7643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:03.863798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84126e-429c-4232-bc31-5d47197b7643 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:03.864099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84126e-429c-4232-bc31-5d47197b7643 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:03.866252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84126e-429c-4232-bc31-5d47197b7643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:03.871821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84126e-429c-4232-bc31-5d47197b7643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:03.873383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84126e-429c-4232-bc31-5d47197b7643 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ff84126e-429c-4232-bc31-5d47197b7643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:03.888259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:03.889161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:03.890128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:03.890128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:03.890128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:03.890128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:03.890128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:03.890128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:03.890128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:03.890128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:03.890128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 294/591] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:03.891153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84126e-429c-4232-bc31-5d47197b7643 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:03.965482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:03.969810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:03.988465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac06bf-2792-4f68-a722-c79df8e9f041 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:04.011330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84126e-429c-4232-bc31-5d47197b7643 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:04.026806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9116a63-4ad2-4b30-8a97-9a2b5ce7cb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:04.027569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9116a63-4ad2-4b30-8a97-9a2b5ce7cb98 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:04.027920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9116a63-4ad2-4b30-8a97-9a2b5ce7cb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:04.028285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9116a63-4ad2-4b30-8a97-9a2b5ce7cb98 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:04.029235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9116a63-4ad2-4b30-8a97-9a2b5ce7cb98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:04.029694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9116a63-4ad2-4b30-8a97-9a2b5ce7cb98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:04.031107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9116a63-4ad2-4b30-8a97-9a2b5ce7cb98 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:04.057256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 298/592] 10.222.0.45 () {56 vars in 1137 bytes} [Tue Jul 5 15:51:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:04.057929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9116a63-4ad2-4b30-8a97-9a2b5ce7cb98 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:04.060886 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330c33e-3028-49fd-bfce-cd5ddf342140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:04.061557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330c33e-3028-49fd-bfce-cd5ddf342140 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:04.061864 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330c33e-3028-49fd-bfce-cd5ddf342140 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:04.062201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330c33e-3028-49fd-bfce-cd5ddf342140 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:04.062838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330c33e-3028-49fd-bfce-cd5ddf342140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:04.063145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330c33e-3028-49fd-bfce-cd5ddf342140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:04.064596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330c33e-3028-49fd-bfce-cd5ddf342140 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:04.077701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 295/593] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:04.078449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330c33e-3028-49fd-bfce-cd5ddf342140 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:04.175969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9116a63-4ad2-4b30-8a97-9a2b5ce7cb98 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:04.206260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330c33e-3028-49fd-bfce-cd5ddf342140 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:04.211851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54429da0-d6db-4a17-afe7-7ea15e0d165a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:04.212511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54429da0-d6db-4a17-afe7-7ea15e0d165a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:04.212812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54429da0-d6db-4a17-afe7-7ea15e0d165a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:04.213095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54429da0-d6db-4a17-afe7-7ea15e0d165a None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:04.213889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54429da0-d6db-4a17-afe7-7ea15e0d165a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:04.214203 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54429da0-d6db-4a17-afe7-7ea15e0d165a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:04.215627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54429da0-d6db-4a17-afe7-7ea15e0d165a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:04.233147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 299/594] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:04.234111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54429da0-d6db-4a17-afe7-7ea15e0d165a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:04.237418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:04.247707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:04.252453 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7963e-92f4-49ff-86e4-85778971cf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:04.253967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7963e-92f4-49ff-86e4-85778971cf4a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:04.254468 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7963e-92f4-49ff-86e4-85778971cf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:04.255128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7963e-92f4-49ff-86e4-85778971cf4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:04.270700 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7963e-92f4-49ff-86e4-85778971cf4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:04.272160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7963e-92f4-49ff-86e4-85778971cf4a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-07f7963e-92f4-49ff-86e4-85778971cf4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:04.302359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:04.303268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:04.304153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:04.304153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:04.304153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:04.304153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:04.308358 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 296/595] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:04.309323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7963e-92f4-49ff-86e4-85778971cf4a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:04.361741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54429da0-d6db-4a17-afe7-7ea15e0d165a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:04.398869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800dfa2-a1ff-4820-a1fe-b666d19f45d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:04.399592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800dfa2-a1ff-4820-a1fe-b666d19f45d4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:04.399924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800dfa2-a1ff-4820-a1fe-b666d19f45d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:04.400205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800dfa2-a1ff-4820-a1fe-b666d19f45d4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:04.408381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800dfa2-a1ff-4820-a1fe-b666d19f45d4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:04.410522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800dfa2-a1ff-4820-a1fe-b666d19f45d4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-c800dfa2-a1ff-4820-a1fe-b666d19f45d4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:04.425878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:04.426798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:04.427740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 300/596] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:04.428923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800dfa2-a1ff-4820-a1fe-b666d19f45d4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:04.436906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7963e-92f4-49ff-86e4-85778971cf4a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:04.475778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f207a59-a722-4d62-979d-4f7d6bb9ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:04.477821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f207a59-a722-4d62-979d-4f7d6bb9ecaf None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:04.478331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f207a59-a722-4d62-979d-4f7d6bb9ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:04.479237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f207a59-a722-4d62-979d-4f7d6bb9ecaf None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/805a9dd2d0934359b7ca5fe629b96d38/roles/ac08a857de8b484fb5dfc65cd4cdc044` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:04.499574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f207a59-a722-4d62-979d-4f7d6bb9ecaf None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=805a9dd2d0934359b7ca5fe629b96d38, role_id=ac08a857de8b484fb5dfc65cd4cdc044)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:04.501353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f207a59-a722-4d62-979d-4f7d6bb9ecaf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:04.506768 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:04.543250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800dfa2-a1ff-4820-a1fe-b666d19f45d4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:04.584257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61265f58-f528-4f8f-95a4-2cb6bd121696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:04.584257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61265f58-f528-4f8f-95a4-2cb6bd121696 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:04.584257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61265f58-f528-4f8f-95a4-2cb6bd121696 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:04.584257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61265f58-f528-4f8f-95a4-2cb6bd121696 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:04.584669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61265f58-f528-4f8f-95a4-2cb6bd121696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:04.584669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61265f58-f528-4f8f-95a4-2cb6bd121696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:04.585232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 297/597] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:51:04 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/805a9dd2d0934359b7ca5fe629b96d38/roles/ac08a857de8b484fb5dfc65cd4cdc044 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:51:04.585391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61265f58-f528-4f8f-95a4-2cb6bd121696 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:04.586612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f207a59-a722-4d62-979d-4f7d6bb9ecaf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:04.618041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 301/598] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:04.622393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61265f58-f528-4f8f-95a4-2cb6bd121696 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:04.656219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:04.793997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f207a59-a722-4d62-979d-4f7d6bb9ecaf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:04.810386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61265f58-f528-4f8f-95a4-2cb6bd121696 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:04.829999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c66416-c57d-4f6b-bd7a-1363aaffa5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:04.830702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c66416-c57d-4f6b-bd7a-1363aaffa5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:04.831005 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c66416-c57d-4f6b-bd7a-1363aaffa5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:04.831542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c66416-c57d-4f6b-bd7a-1363aaffa5f9 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:04.832115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c66416-c57d-4f6b-bd7a-1363aaffa5f9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:04.832427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c66416-c57d-4f6b-bd7a-1363aaffa5f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:04.833907 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c66416-c57d-4f6b-bd7a-1363aaffa5f9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:04.842400 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:04.854598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 298/599] 10.222.0.45 () {56 vars in 1139 bytes} [Tue Jul 5 15:51:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:04.857063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ce240-688c-4c47-b0a3-a785d3a521af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:04.858163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ce240-688c-4c47-b0a3-a785d3a521af None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:04.858586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ce240-688c-4c47-b0a3-a785d3a521af None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:04.858996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ce240-688c-4c47-b0a3-a785d3a521af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:04.860417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c66416-c57d-4f6b-bd7a-1363aaffa5f9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:04.868479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ce240-688c-4c47-b0a3-a785d3a521af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:04.870247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ce240-688c-4c47-b0a3-a785d3a521af None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-827ce240-688c-4c47-b0a3-a785d3a521af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:04.886653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:04.887954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:04.889142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:04.889142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:04.889142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:04.889142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:04.889142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:04.889142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:04.889142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:04.889142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:04.891948 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 302/600] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:04.901841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ce240-688c-4c47-b0a3-a785d3a521af None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:04.989263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c66416-c57d-4f6b-bd7a-1363aaffa5f9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:05.017149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:05.052094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ce240-688c-4c47-b0a3-a785d3a521af None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:05.062072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd12168-5c65-4138-b5f1-85e0352eacef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:05.062885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd12168-5c65-4138-b5f1-85e0352eacef None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.063210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd12168-5c65-4138-b5f1-85e0352eacef None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.063532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd12168-5c65-4138-b5f1-85e0352eacef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.074549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd12168-5c65-4138-b5f1-85e0352eacef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:05.076041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd12168-5c65-4138-b5f1-85e0352eacef None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:05.088205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122532f-6c85-4706-9de5-d381a9d97a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:05.088902 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122532f-6c85-4706-9de5-d381a9d97a71 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.089185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122532f-6c85-4706-9de5-d381a9d97a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.089571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122532f-6c85-4706-9de5-d381a9d97a71 None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.090254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122532f-6c85-4706-9de5-d381a9d97a71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:05.090563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122532f-6c85-4706-9de5-d381a9d97a71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:05.091994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122532f-6c85-4706-9de5-d381a9d97a71 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:05.105716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 303/601] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-afd12168-5c65-4138-b5f1-85e0352eacef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:05.114168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:05.115072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:05.116245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:05.116245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:05.116245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:05.116245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:05.116245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 299/602] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:05.116555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122532f-6c85-4706-9de5-d381a9d97a71 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:05.122174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd12168-5c65-4138-b5f1-85e0352eacef None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:05.242575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd12168-5c65-4138-b5f1-85e0352eacef None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:05.253988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:05.293668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693500ed-2728-4e80-b402-1a48ad7fd200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:05.295020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693500ed-2728-4e80-b402-1a48ad7fd200 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.295639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693500ed-2728-4e80-b402-1a48ad7fd200 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.296201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693500ed-2728-4e80-b402-1a48ad7fd200 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.297289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693500ed-2728-4e80-b402-1a48ad7fd200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:05.298102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693500ed-2728-4e80-b402-1a48ad7fd200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:05.299954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693500ed-2728-4e80-b402-1a48ad7fd200 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:05.300492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122532f-6c85-4706-9de5-d381a9d97a71 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:05.315887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 300/603] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:05.320510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6223a0f-21a2-4199-8bec-992f61dec707 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.320510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6223a0f-21a2-4199-8bec-992f61dec707 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.320510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6223a0f-21a2-4199-8bec-992f61dec707 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.321571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 301/604] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:05.321942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:05.324928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6223a0f-21a2-4199-8bec-992f61dec707 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:05.339084 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291a6a4-da8f-425f-8b53-4910c77b222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:05.339971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291a6a4-da8f-425f-8b53-4910c77b222b None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.340327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291a6a4-da8f-425f-8b53-4910c77b222b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.340611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291a6a4-da8f-425f-8b53-4910c77b222b None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/a5b712bd7d8d4930a3bfc99581be8cb7/roles/ac08a857de8b484fb5dfc65cd4cdc044` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.358719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291a6a4-da8f-425f-8b53-4910c77b222b None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=a5b712bd7d8d4930a3bfc99581be8cb7, role_id=ac08a857de8b484fb5dfc65cd4cdc044)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:05.360912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291a6a4-da8f-425f-8b53-4910c77b222b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:05.442809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6223a0f-21a2-4199-8bec-992f61dec707 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:05.464973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 304/605] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:51:05 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/a5b712bd7d8d4930a3bfc99581be8cb7/roles/ac08a857de8b484fb5dfc65cd4cdc044 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:51:05.467072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e472e-3cb3-42ea-bf7b-51f75ee70ca0 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.467694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e472e-3cb3-42ea-bf7b-51f75ee70ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.468194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e472e-3cb3-42ea-bf7b-51f75ee70ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.479055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f7437-6d53-4cf7-a876-dcc459e7f690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:05.479849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f7437-6d53-4cf7-a876-dcc459e7f690 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.480150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f7437-6d53-4cf7-a876-dcc459e7f690 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.480407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f7437-6d53-4cf7-a876-dcc459e7f690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.488165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f7437-6d53-4cf7-a876-dcc459e7f690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:05.489607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f7437-6d53-4cf7-a876-dcc459e7f690 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-358f7437-6d53-4cf7-a876-dcc459e7f690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:05.504691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:05.505683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:05.506582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:05.506582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:05.506582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:05.506582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:05.506582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:05.506582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:05.506582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:05.506582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:05.506582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 302/606] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:05.508694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d33e472e-3cb3-42ea-bf7b-51f75ee70ca0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:05.510228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d33e472e-3cb3-42ea-bf7b-51f75ee70ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:05.512900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f7437-6d53-4cf7-a876-dcc459e7f690 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:05.628182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:05.636805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 305/607] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:05.636805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382be5-5faa-4d0f-814d-858ce4d51a20 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.636805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382be5-5faa-4d0f-814d-858ce4d51a20 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.636805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382be5-5faa-4d0f-814d-858ce4d51a20 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.643955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 306/608] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:05.649456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3a4a5-9ecc-4d41-bb50-2ffe00fb94d9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.649739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3a4a5-9ecc-4d41-bb50-2ffe00fb94d9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.650016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3a4a5-9ecc-4d41-bb50-2ffe00fb94d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.671275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-08e3a4a5-9ecc-4d41-bb50-2ffe00fb94d9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:05.672487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-08e3a4a5-9ecc-4d41-bb50-2ffe00fb94d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:05.679916 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f7437-6d53-4cf7-a876-dcc459e7f690 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:05.691685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:05.716985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6cd02-8706-4ba7-9a16-4e6b60eed2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:05.717960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6cd02-8706-4ba7-9a16-4e6b60eed2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.718456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6cd02-8706-4ba7-9a16-4e6b60eed2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.718931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6cd02-8706-4ba7-9a16-4e6b60eed2d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.719986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6cd02-8706-4ba7-9a16-4e6b60eed2d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:05.720534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6cd02-8706-4ba7-9a16-4e6b60eed2d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:05.722266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6cd02-8706-4ba7-9a16-4e6b60eed2d5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:05.740814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 303/609] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:05.740814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6cd02-8706-4ba7-9a16-4e6b60eed2d5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:05.753659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 307/610] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:05.754716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3a4a5-9ecc-4d41-bb50-2ffe00fb94d9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:05.867460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3a4a5-9ecc-4d41-bb50-2ffe00fb94d9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:05.882675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6cd02-8706-4ba7-9a16-4e6b60eed2d5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:05.908463 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea9e71-aae6-41e3-8557-fb5f7856615d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:05.909591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea9e71-aae6-41e3-8557-fb5f7856615d None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.909976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:05.910838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea9e71-aae6-41e3-8557-fb5f7856615d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.911194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea9e71-aae6-41e3-8557-fb5f7856615d None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/887d2ad3cbf7459a8a59e7442ba9744e/roles/ac08a857de8b484fb5dfc65cd4cdc044` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.929582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea9e71-aae6-41e3-8557-fb5f7856615d None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=887d2ad3cbf7459a8a59e7442ba9744e, role_id=ac08a857de8b484fb5dfc65cd4cdc044)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:05.931014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea9e71-aae6-41e3-8557-fb5f7856615d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:05.935099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e85712-c0bc-4778-b1cc-10633be6f860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:05.936235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e85712-c0bc-4778-b1cc-10633be6f860 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:05.936746 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e85712-c0bc-4778-b1cc-10633be6f860 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:05.937271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e85712-c0bc-4778-b1cc-10633be6f860 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:05.948677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e85712-c0bc-4778-b1cc-10633be6f860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b1e85712-c0bc-4778-b1cc-10633be6f860 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:05.956747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:05.957798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:05.958634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:05.959935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 304/611] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:05.964420 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e85712-c0bc-4778-b1cc-10633be6f860 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:06.013355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 308/612] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:51:05 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/887d2ad3cbf7459a8a59e7442ba9744e/roles/ac08a857de8b484fb5dfc65cd4cdc044 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:51:06.015253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea9e71-aae6-41e3-8557-fb5f7856615d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:06.022037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:06.118957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e85712-c0bc-4778-b1cc-10633be6f860 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:06.160461 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2f943-1ec4-4dca-a1ee-bb561b6c3bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:06.161681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2f943-1ec4-4dca-a1ee-bb561b6c3bfe None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:06.162177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2f943-1ec4-4dca-a1ee-bb561b6c3bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:06.162648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2f943-1ec4-4dca-a1ee-bb561b6c3bfe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:06.176118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2f943-1ec4-4dca-a1ee-bb561b6c3bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-80e2f943-1ec4-4dca-a1ee-bb561b6c3bfe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:06.185664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:06.186620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:06.187446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:06.190813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 305/613] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:06.198655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2f943-1ec4-4dca-a1ee-bb561b6c3bfe None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:06.237765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea9e71-aae6-41e3-8557-fb5f7856615d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:06.266165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:06.278533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4f583-7125-4cce-8912-51be30e7b3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:06.280909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4f583-7125-4cce-8912-51be30e7b3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:06.280909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4f583-7125-4cce-8912-51be30e7b3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:06.280909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4f583-7125-4cce-8912-51be30e7b3b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:06.280909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4f583-7125-4cce-8912-51be30e7b3b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:06.281974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4f583-7125-4cce-8912-51be30e7b3b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:06.283193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4f583-7125-4cce-8912-51be30e7b3b9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:06.293460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 309/614] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:06.300093 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4f583-7125-4cce-8912-51be30e7b3b9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:06.350440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2f943-1ec4-4dca-a1ee-bb561b6c3bfe None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:06.385088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:06.396501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b8ee-2e86-4576-902e-e03f2067da5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:06.397414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b8ee-2e86-4576-902e-e03f2067da5c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:06.397859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b8ee-2e86-4576-902e-e03f2067da5c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:06.398295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b8ee-2e86-4576-902e-e03f2067da5c None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:06.399180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616b8ee-2e86-4576-902e-e03f2067da5c None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:06.399624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616b8ee-2e86-4576-902e-e03f2067da5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:06.401165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616b8ee-2e86-4576-902e-e03f2067da5c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:06.410546 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 306/615] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:06.416956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b8ee-2e86-4576-902e-e03f2067da5c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:06.433621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4f583-7125-4cce-8912-51be30e7b3b9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:06.478820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7c4fd-a34d-4388-b02d-9da017b836a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:06.480159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7c4fd-a34d-4388-b02d-9da017b836a4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:06.480656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7c4fd-a34d-4388-b02d-9da017b836a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:06.481170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7c4fd-a34d-4388-b02d-9da017b836a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:06.492845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7c4fd-a34d-4388-b02d-9da017b836a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:06.494542 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7c4fd-a34d-4388-b02d-9da017b836a4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-56c7c4fd-a34d-4388-b02d-9da017b836a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:06.512851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:06.513763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:06.514616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:06.514616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:06.514616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:06.514616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:06.521166 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 310/616] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:06.523178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b8ee-2e86-4576-902e-e03f2067da5c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:06.523464 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7c4fd-a34d-4388-b02d-9da017b836a4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:06.524879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:06.558593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b2829-6353-4db3-81c8-1f260d715376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:06.565355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b2829-6353-4db3-81c8-1f260d715376 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:06.565355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b2829-6353-4db3-81c8-1f260d715376 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:06.565355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b2829-6353-4db3-81c8-1f260d715376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:06.570553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b2829-6353-4db3-81c8-1f260d715376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:06.572017 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b2829-6353-4db3-81c8-1f260d715376 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3a9b2829-6353-4db3-81c8-1f260d715376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:06.590538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:06.591442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:06.592328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:06.592328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:06.592328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:06.592328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:06.592328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 307/617] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:06.598020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b2829-6353-4db3-81c8-1f260d715376 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:06.703705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b2829-6353-4db3-81c8-1f260d715376 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:06.739369 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338d305-dcf1-4bdd-9003-8c4f4c9cf9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:06.740039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338d305-dcf1-4bdd-9003-8c4f4c9cf9ad None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:06.740323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338d305-dcf1-4bdd-9003-8c4f4c9cf9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:06.740601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338d305-dcf1-4bdd-9003-8c4f4c9cf9ad None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:06.741008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7c4fd-a34d-4388-b02d-9da017b836a4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:06.741355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338d305-dcf1-4bdd-9003-8c4f4c9cf9ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:06.741662 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338d305-dcf1-4bdd-9003-8c4f4c9cf9ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:06.743092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338d305-dcf1-4bdd-9003-8c4f4c9cf9ad None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:06.755386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 308/618] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:06.759973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:06.761708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338d305-dcf1-4bdd-9003-8c4f4c9cf9ad None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:06.784866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:06.785044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0e540-231c-4124-ad25-e0aee2282524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:06.785044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0e540-231c-4124-ad25-e0aee2282524 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:06.785044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0e540-231c-4124-ad25-e0aee2282524 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:06.785044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0e540-231c-4124-ad25-e0aee2282524 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:06.785414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0e540-231c-4124-ad25-e0aee2282524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:06.785414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0e540-231c-4124-ad25-e0aee2282524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:06.785414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0e540-231c-4124-ad25-e0aee2282524 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:06.805600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 311/619] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:06.811243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0e540-231c-4124-ad25-e0aee2282524 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:06.881702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338d305-dcf1-4bdd-9003-8c4f4c9cf9ad None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:06.919675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498df7c4-6c51-4c06-800b-c005d11eab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:06.920569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498df7c4-6c51-4c06-800b-c005d11eab66 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:06.920929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498df7c4-6c51-4c06-800b-c005d11eab66 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:06.921209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498df7c4-6c51-4c06-800b-c005d11eab66 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:06.929228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498df7c4-6c51-4c06-800b-c005d11eab66 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:06.930786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498df7c4-6c51-4c06-800b-c005d11eab66 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-498df7c4-6c51-4c06-800b-c005d11eab66 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:06.947114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:06.948090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:06.949071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 309/620] 10.222.0.45 () {56 vars in 1093 bytes} [Tue Jul 5 15:51:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:06.954243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498df7c4-6c51-4c06-800b-c005d11eab66 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:07.000592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0e540-231c-4124-ad25-e0aee2282524 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:07.035682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:07.038545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a34e4-9fdd-4c20-876a-14ffb7e658e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:07.039535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a34e4-9fdd-4c20-876a-14ffb7e658e6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.039983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a34e4-9fdd-4c20-876a-14ffb7e658e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.040474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a34e4-9fdd-4c20-876a-14ffb7e658e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.048068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a34e4-9fdd-4c20-876a-14ffb7e658e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:07.050038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a34e4-9fdd-4c20-876a-14ffb7e658e6 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:07.063162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498df7c4-6c51-4c06-800b-c005d11eab66 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-c54a34e4-9fdd-4c20-876a-14ffb7e658e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:07.067322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:07.068441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:07.069580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:07.069580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:07.069580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:07.069580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:07.069580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:07.069580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:07.069580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:07.069580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:07.069580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:07.069580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:07.072785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 312/621] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:07.074699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6c54b-3a7f-47cf-8cce-20e6118f72fc None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.075265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6c54b-3a7f-47cf-8cce-20e6118f72fc None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.075746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6c54b-3a7f-47cf-8cce-20e6118f72fc None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.081949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 313/622] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:07.083038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6c54b-3a7f-47cf-8cce-20e6118f72fc None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:07.100781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8811774-e826-4a4b-b3b7-0ed18f27c5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:07.101461 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8811774-e826-4a4b-b3b7-0ed18f27c5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.102011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8811774-e826-4a4b-b3b7-0ed18f27c5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.102290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8811774-e826-4a4b-b3b7-0ed18f27c5e3 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.103131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8811774-e826-4a4b-b3b7-0ed18f27c5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:07.103442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8811774-e826-4a4b-b3b7-0ed18f27c5e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:07.105137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8811774-e826-4a4b-b3b7-0ed18f27c5e3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:07.122740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:07.125488 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 310/623] 10.222.0.45 () {56 vars in 1133 bytes} [Tue Jul 5 15:51:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:07.126766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f44aac-5647-4019-a690-c67fec2f8ed3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.127070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f44aac-5647-4019-a690-c67fec2f8ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.132169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f44aac-5647-4019-a690-c67fec2f8ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.149151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-00f44aac-5647-4019-a690-c67fec2f8ed3 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:07.150337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-00f44aac-5647-4019-a690-c67fec2f8ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:07.197713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6c54b-3a7f-47cf-8cce-20e6118f72fc None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:07.218425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 311/624] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:07.219258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f44aac-5647-4019-a690-c67fec2f8ed3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:07.244223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f495c0-00f2-4589-8387-c335d56294e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:07.245021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f495c0-00f2-4589-8387-c335d56294e9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.245360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f495c0-00f2-4589-8387-c335d56294e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.245640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f495c0-00f2-4589-8387-c335d56294e9 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.246582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f495c0-00f2-4589-8387-c335d56294e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:07.246991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f495c0-00f2-4589-8387-c335d56294e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:07.248706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f495c0-00f2-4589-8387-c335d56294e9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:07.278064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 314/625] 10.222.0.45 () {56 vars in 1135 bytes} [Tue Jul 5 15:51:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:07.279072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f495c0-00f2-4589-8387-c335d56294e9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:07.281915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:07.321450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f44aac-5647-4019-a690-c67fec2f8ed3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:07.357010 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d676f-255e-497b-a3fb-a6a6b54a6b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:07.357665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d676f-255e-497b-a3fb-a6a6b54a6b26 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.357949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d676f-255e-497b-a3fb-a6a6b54a6b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.358224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d676f-255e-497b-a3fb-a6a6b54a6b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.389973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:07.397721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f495c0-00f2-4589-8387-c335d56294e9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:07.429332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d676f-255e-497b-a3fb-a6a6b54a6b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:07.430798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d676f-255e-497b-a3fb-a6a6b54a6b26 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:07.436306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa2be7-6b06-4a9d-8de3-85e2522a3a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:07.437342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa2be7-6b06-4a9d-8de3-85e2522a3a23 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.437789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa2be7-6b06-4a9d-8de3-85e2522a3a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.438252 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa2be7-6b06-4a9d-8de3-85e2522a3a23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.450537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa2be7-6b06-4a9d-8de3-85e2522a3a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-b7fa2be7-6b06-4a9d-8de3-85e2522a3a23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:07.458961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:07.460102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:07.461136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:07.468203 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 315/626] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:07.469259 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa2be7-6b06-4a9d-8de3-85e2522a3a23 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:07.496081 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:07.543586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 312/627] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:51:07.544406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d676f-255e-497b-a3fb-a6a6b54a6b26 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:07.582906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa2be7-6b06-4a9d-8de3-85e2522a3a23 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:07.620915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b22c23-ee20-4b4f-8de5-2f138be4189d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:07.622045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b22c23-ee20-4b4f-8de5-2f138be4189d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.622474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b22c23-ee20-4b4f-8de5-2f138be4189d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.622920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b22c23-ee20-4b4f-8de5-2f138be4189d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.634593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b22c23-ee20-4b4f-8de5-2f138be4189d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:07.636326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b22c23-ee20-4b4f-8de5-2f138be4189d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:07.647711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d676f-255e-497b-a3fb-a6a6b54a6b26 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-08b22c23-ee20-4b4f-8de5-2f138be4189d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:07.654806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:07.655911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:07.657021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:07.657021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:07.657021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:07.657021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:07.659574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 316/628] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:07.660500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b22c23-ee20-4b4f-8de5-2f138be4189d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:07.670356 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:07.684071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e361f4-50b8-41e8-a7c7-5dcb9422c320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:07.684745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e361f4-50b8-41e8-a7c7-5dcb9422c320 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.685068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e361f4-50b8-41e8-a7c7-5dcb9422c320 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.685347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e361f4-50b8-41e8-a7c7-5dcb9422c320 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.686095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e361f4-50b8-41e8-a7c7-5dcb9422c320 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:07.686402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e361f4-50b8-41e8-a7c7-5dcb9422c320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:07.687771 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e361f4-50b8-41e8-a7c7-5dcb9422c320 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:07.696292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 313/629] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:07.701481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e361f4-50b8-41e8-a7c7-5dcb9422c320 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:07.727684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:07.773462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b22c23-ee20-4b4f-8de5-2f138be4189d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:07.806912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e361f4-50b8-41e8-a7c7-5dcb9422c320 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:07.816564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847e01-4716-4c65-8fc8-6aee24a9f0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:07.817435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847e01-4716-4c65-8fc8-6aee24a9f0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.817719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847e01-4716-4c65-8fc8-6aee24a9f0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.817996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847e01-4716-4c65-8fc8-6aee24a9f0d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.831846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a847e01-4716-4c65-8fc8-6aee24a9f0d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:07.833629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a847e01-4716-4c65-8fc8-6aee24a9f0d9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:07.847959 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f1d5b-9223-4cde-84d3-facb7352f8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:07.848969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f1d5b-9223-4cde-84d3-facb7352f8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:07.849448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f1d5b-9223-4cde-84d3-facb7352f8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:07.849909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f1d5b-9223-4cde-84d3-facb7352f8b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:07.850803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f1d5b-9223-4cde-84d3-facb7352f8b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:07.851291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f1d5b-9223-4cde-84d3-facb7352f8b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:07.853121 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f1d5b-9223-4cde-84d3-facb7352f8b1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-8a847e01-4716-4c65-8fc8-6aee24a9f0d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:07.856047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:07.856937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:07.857812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:07.857812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:07.857812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:07.857812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:07.857812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 317/630] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:07.858482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847e01-4716-4c65-8fc8-6aee24a9f0d9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:07.874623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 314/631] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:07.874623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f1d5b-9223-4cde-84d3-facb7352f8b1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:07.936165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:07.969797 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:07.983469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847e01-4716-4c65-8fc8-6aee24a9f0d9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:07.986755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f1d5b-9223-4cde-84d3-facb7352f8b1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:08.020991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f65f6-7b04-40e9-bffa-f3be637fcedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.021779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f65f6-7b04-40e9-bffa-f3be637fcedd None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.022067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f65f6-7b04-40e9-bffa-f3be637fcedd None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.022294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d1e3e-3184-451b-8a21-c762fd610579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.022405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f65f6-7b04-40e9-bffa-f3be637fcedd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.022950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d1e3e-3184-451b-8a21-c762fd610579 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.023234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d1e3e-3184-451b-8a21-c762fd610579 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.023511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d1e3e-3184-451b-8a21-c762fd610579 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.024241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d1e3e-3184-451b-8a21-c762fd610579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:08.024546 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d1e3e-3184-451b-8a21-c762fd610579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:08.025998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d1e3e-3184-451b-8a21-c762fd610579 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:08.034447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f65f6-7b04-40e9-bffa-f3be637fcedd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:08.036164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f65f6-7b04-40e9-bffa-f3be637fcedd None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:08.038311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 315/632] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:08.039118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d1e3e-3184-451b-8a21-c762fd610579 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-ee8f65f6-7b04-40e9-bffa-f3be637fcedd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:08.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:08.058979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:08.060011 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:08.060011 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:08.060011 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:08.060011 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:08.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 318/633] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:08.063035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ec762-e144-4988-b595-c90c6d6fdb1e None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.063322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ec762-e144-4988-b595-c90c6d6fdb1e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.063601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ec762-e144-4988-b595-c90c6d6fdb1e None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.065051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 319/634] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:08.069837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ec762-e144-4988-b595-c90c6d6fdb1e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:08.150435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d1e3e-3184-451b-8a21-c762fd610579 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:08.177730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ec762-e144-4988-b595-c90c6d6fdb1e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:08.185181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7a582-82ac-4b11-b72b-e3e46cd96421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.185965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7a582-82ac-4b11-b72b-e3e46cd96421 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.186246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7a582-82ac-4b11-b72b-e3e46cd96421 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.186519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7a582-82ac-4b11-b72b-e3e46cd96421 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.189627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:08.193377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7a582-82ac-4b11-b72b-e3e46cd96421 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:08.194917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7a582-82ac-4b11-b72b-e3e46cd96421 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-46c7a582-82ac-4b11-b72b-e3e46cd96421 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:08.209902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:08.210872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:08.211769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:08.211769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:08.211769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:08.211769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:08.211769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:08.211769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:08.211769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:08.211769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:08.211769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 316/635] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:08.212968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7a582-82ac-4b11-b72b-e3e46cd96421 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:08.216330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:08.221226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39290e15-6231-4ad2-939d-11fc9164e397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.222543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39290e15-6231-4ad2-939d-11fc9164e397 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.222883 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39290e15-6231-4ad2-939d-11fc9164e397 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.223162 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39290e15-6231-4ad2-939d-11fc9164e397 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.230942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39290e15-6231-4ad2-939d-11fc9164e397 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:08.232944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39290e15-6231-4ad2-939d-11fc9164e397 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-39290e15-6231-4ad2-939d-11fc9164e397 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:08.249095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:08.250012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:08.250879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:08.250879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:08.250879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:08.250879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:08.250879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:08.250879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:08.250879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:08.250879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:08.255068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 320/636] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:08.256179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849df69f-3f62-484d-8b2c-19dd02298013 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.256462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849df69f-3f62-484d-8b2c-19dd02298013 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.256734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849df69f-3f62-484d-8b2c-19dd02298013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.277579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-849df69f-3f62-484d-8b2c-19dd02298013 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:08.278808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-849df69f-3f62-484d-8b2c-19dd02298013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:08.338232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7a582-82ac-4b11-b72b-e3e46cd96421 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:08.349309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 321/637] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:08.350141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849df69f-3f62-484d-8b2c-19dd02298013 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:08.374456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42406cd8-237c-4465-bb63-f2466286e667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.375354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42406cd8-237c-4465-bb63-f2466286e667 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.375788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42406cd8-237c-4465-bb63-f2466286e667 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.376233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42406cd8-237c-4465-bb63-f2466286e667 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.377215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42406cd8-237c-4465-bb63-f2466286e667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:08.378196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42406cd8-237c-4465-bb63-f2466286e667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:08.379597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42406cd8-237c-4465-bb63-f2466286e667 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:08.392041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 317/638] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:08.393786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42406cd8-237c-4465-bb63-f2466286e667 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:08.458717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:08.463718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:08.466733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849df69f-3f62-484d-8b2c-19dd02298013 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:08.496444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42406cd8-237c-4465-bb63-f2466286e667 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:08.499202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1b6d6-eb8a-4944-913d-4757e020e351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.499853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1b6d6-eb8a-4944-913d-4757e020e351 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.500139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1b6d6-eb8a-4944-913d-4757e020e351 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.500415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1b6d6-eb8a-4944-913d-4757e020e351 None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.501240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1b6d6-eb8a-4944-913d-4757e020e351 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:08.501552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1b6d6-eb8a-4944-913d-4757e020e351 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:08.503653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1b6d6-eb8a-4944-913d-4757e020e351 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:08.521069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 322/639] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:08.521069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1b6d6-eb8a-4944-913d-4757e020e351 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:08.535118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1381b-0566-4f8b-9a55-99f8acfd3666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.535772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1381b-0566-4f8b-9a55-99f8acfd3666 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.536054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1381b-0566-4f8b-9a55-99f8acfd3666 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.536329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1381b-0566-4f8b-9a55-99f8acfd3666 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.537143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1381b-0566-4f8b-9a55-99f8acfd3666 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:08.537472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1381b-0566-4f8b-9a55-99f8acfd3666 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:08.538868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1381b-0566-4f8b-9a55-99f8acfd3666 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:08.550528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 318/640] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:08.551286 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1381b-0566-4f8b-9a55-99f8acfd3666 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:08.618852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1b6d6-eb8a-4944-913d-4757e020e351 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:08.647878 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1381b-0566-4f8b-9a55-99f8acfd3666 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:08.652437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baefbe-e1d6-4642-8665-a63d15f4332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.653304 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baefbe-e1d6-4642-8665-a63d15f4332f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.653590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baefbe-e1d6-4642-8665-a63d15f4332f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.653864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baefbe-e1d6-4642-8665-a63d15f4332f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.664062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baefbe-e1d6-4642-8665-a63d15f4332f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:08.670510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-b8baefbe-e1d6-4642-8665-a63d15f4332f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:08.671673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:08.672632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:08.673541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 323/641] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:08.674481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccaf6b-153f-46b7-b4cf-4f590ad3d2ed None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.674818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccaf6b-153f-46b7-b4cf-4f590ad3d2ed None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.674943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccaf6b-153f-46b7-b4cf-4f590ad3d2ed None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.676198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 324/642] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:08.676898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccaf6b-153f-46b7-b4cf-4f590ad3d2ed None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:08.686483 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ce4f6-da0a-496b-bc10-5de1c878c332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.687264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ce4f6-da0a-496b-bc10-5de1c878c332 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.687542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ce4f6-da0a-496b-bc10-5de1c878c332 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.687816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ce4f6-da0a-496b-bc10-5de1c878c332 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.695359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ce4f6-da0a-496b-bc10-5de1c878c332 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:08.696812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ce4f6-da0a-496b-bc10-5de1c878c332 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:08.698963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-056ce4f6-da0a-496b-bc10-5de1c878c332 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:08.711973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:08.718656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 319/643] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:08.719875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ce4f6-da0a-496b-bc10-5de1c878c332 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:08.783772 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccaf6b-153f-46b7-b4cf-4f590ad3d2ed None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:08.812008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ce4f6-da0a-496b-bc10-5de1c878c332 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:08.817859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256ca09-45a2-46c8-aa05-1bcd678149e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.818734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256ca09-45a2-46c8-aa05-1bcd678149e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.819011 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256ca09-45a2-46c8-aa05-1bcd678149e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.819284 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256ca09-45a2-46c8-aa05-1bcd678149e2 None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/8da9cdd70af646928ee0edcadbcb524f/roles/ac08a857de8b484fb5dfc65cd4cdc044` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.834413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256ca09-45a2-46c8-aa05-1bcd678149e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=8da9cdd70af646928ee0edcadbcb524f, role_id=ac08a857de8b484fb5dfc65cd4cdc044)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:08.835877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256ca09-45a2-46c8-aa05-1bcd678149e2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:08.846299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7edeb5-1e0d-4b29-994d-980dc1093dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:08.846943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7edeb5-1e0d-4b29-994d-980dc1093dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.847221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7edeb5-1e0d-4b29-994d-980dc1093dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.847492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7edeb5-1e0d-4b29-994d-980dc1093dd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.848317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7edeb5-1e0d-4b29-994d-980dc1093dd7 None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=ef95ab7b924d456195a8761e894d911b) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:08.848626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7edeb5-1e0d-4b29-994d-980dc1093dd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:08.850072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7edeb5-1e0d-4b29-994d-980dc1093dd7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:08.855965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 320/644] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:08 2022] GET /identity/v3/role_assignments?user.id=ef95ab7b924d456195a8761e894d911b&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:08.856719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7edeb5-1e0d-4b29-994d-980dc1093dd7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:08.909113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:08.938881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 325/645] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:51:08 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/8da9cdd70af646928ee0edcadbcb524f/roles/ac08a857de8b484fb5dfc65cd4cdc044 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:51:08.943683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7f917-db12-4a6d-85a7-eba969c40e30 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:08.943683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7f917-db12-4a6d-85a7-eba969c40e30 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:08.943683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7f917-db12-4a6d-85a7-eba969c40e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:08.961210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8ed7f917-db12-4a6d-85a7-eba969c40e30 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:08.962933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8ed7f917-db12-4a6d-85a7-eba969c40e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:08.986571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7edeb5-1e0d-4b29-994d-980dc1093dd7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:09.030600 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cae19-1350-4f5c-878d-656f934e37ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:09.031305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cae19-1350-4f5c-878d-656f934e37ee None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:09.031579 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cae19-1350-4f5c-878d-656f934e37ee None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:09.031858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cae19-1350-4f5c-878d-656f934e37ee None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:09.032876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cae19-1350-4f5c-878d-656f934e37ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:09.033197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cae19-1350-4f5c-878d-656f934e37ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:09.037563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cae19-1350-4f5c-878d-656f934e37ee None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:09.048195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 321/646] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:09.050517 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cae19-1350-4f5c-878d-656f934e37ee None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:09.071537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:09.078000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 326/647] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:09.082482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7f917-db12-4a6d-85a7-eba969c40e30 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:09.170670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cae19-1350-4f5c-878d-656f934e37ee None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:09.184135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:09.214209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7f917-db12-4a6d-85a7-eba969c40e30 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:09.217642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0402e75-d468-41a8-b3ab-5ef21f75ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:09.218089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0402e75-d468-41a8-b3ab-5ef21f75ff24 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:09.218376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0402e75-d468-41a8-b3ab-5ef21f75ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:09.218650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0402e75-d468-41a8-b3ab-5ef21f75ff24 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:09.219497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0402e75-d468-41a8-b3ab-5ef21f75ff24 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:09.219811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0402e75-d468-41a8-b3ab-5ef21f75ff24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:09.221835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0402e75-d468-41a8-b3ab-5ef21f75ff24 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:09.243221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 322/648] 10.222.0.45 () {56 vars in 1137 bytes} [Tue Jul 5 15:51:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:09.244090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0402e75-d468-41a8-b3ab-5ef21f75ff24 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:09.259500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee582a4d-6f69-453f-898a-683de6a000fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:09.260304 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee582a4d-6f69-453f-898a-683de6a000fd None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:09.260591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee582a4d-6f69-453f-898a-683de6a000fd None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:09.260900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee582a4d-6f69-453f-898a-683de6a000fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:09.275107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee582a4d-6f69-453f-898a-683de6a000fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:09.276641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee582a4d-6f69-453f-898a-683de6a000fd None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-ee582a4d-6f69-453f-898a-683de6a000fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:09.298938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:09.300424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:09.301376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:09.301376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:09.301376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:09.301376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:09.301376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 327/649] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:09.302211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee582a4d-6f69-453f-898a-683de6a000fd None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:09.365781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:09.379436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0402e75-d468-41a8-b3ab-5ef21f75ff24 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:09.417766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ba17-46a6-4152-b255-f5f1985daa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:09.418810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ba17-46a6-4152-b255-f5f1985daa0d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:09.419310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ba17-46a6-4152-b255-f5f1985daa0d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:09.419781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ba17-46a6-4152-b255-f5f1985daa0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:09.423148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee582a4d-6f69-453f-898a-683de6a000fd None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:09.432106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4ba17-46a6-4152-b255-f5f1985daa0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-11e4ba17-46a6-4152-b255-f5f1985daa0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:09.440277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:09.441285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:09.442108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:09.449031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 323/650] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:09.450135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ba17-46a6-4152-b255-f5f1985daa0d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:09.461575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a22fed-b1d3-4c79-8d74-b626297bcfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:09.462684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a22fed-b1d3-4c79-8d74-b626297bcfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:09.462979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a22fed-b1d3-4c79-8d74-b626297bcfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:09.463306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a22fed-b1d3-4c79-8d74-b626297bcfa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:09.486660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:09.550186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a22fed-b1d3-4c79-8d74-b626297bcfa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:09.552493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a22fed-b1d3-4c79-8d74-b626297bcfa2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:09.569234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ba17-46a6-4152-b255-f5f1985daa0d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:09.615675 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:09.617666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923700c1-9e81-4c3e-8962-d294a3e10acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:09.618332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923700c1-9e81-4c3e-8962-d294a3e10acc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:09.618763 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923700c1-9e81-4c3e-8962-d294a3e10acc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:09.619047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923700c1-9e81-4c3e-8962-d294a3e10acc None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:09.620109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923700c1-9e81-4c3e-8962-d294a3e10acc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:09.620679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923700c1-9e81-4c3e-8962-d294a3e10acc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:09.622559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923700c1-9e81-4c3e-8962-d294a3e10acc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:09.637082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 324/651] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:09.637949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923700c1-9e81-4c3e-8962-d294a3e10acc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:09.669755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 328/652] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:51:09.670619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a22fed-b1d3-4c79-8d74-b626297bcfa2 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:09.743907 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:09.757365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923700c1-9e81-4c3e-8962-d294a3e10acc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:09.781322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a22fed-b1d3-4c79-8d74-b626297bcfa2 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:09.795674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb45aa-1c77-477e-b714-ad7b1ecc1127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:09.796381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb45aa-1c77-477e-b714-ad7b1ecc1127 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:09.796626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb45aa-1c77-477e-b714-ad7b1ecc1127 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:09.796935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb45aa-1c77-477e-b714-ad7b1ecc1127 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:09.797707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb45aa-1c77-477e-b714-ad7b1ecc1127 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:09.798165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb45aa-1c77-477e-b714-ad7b1ecc1127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:09.799952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb45aa-1c77-477e-b714-ad7b1ecc1127 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:09.812704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 325/653] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:09.812704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb45aa-1c77-477e-b714-ad7b1ecc1127 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:09.825764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d653e-bd72-4cd2-adc7-22d1cd77fd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:09.826866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d653e-bd72-4cd2-adc7-22d1cd77fd7b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:09.827356 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d653e-bd72-4cd2-adc7-22d1cd77fd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:09.827842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d653e-bd72-4cd2-adc7-22d1cd77fd7b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:09.835667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d653e-bd72-4cd2-adc7-22d1cd77fd7b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:09.837403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d653e-bd72-4cd2-adc7-22d1cd77fd7b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-626d653e-bd72-4cd2-adc7-22d1cd77fd7b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:09.853269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:09.854187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:09.855196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:09.855196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:09.855196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:09.855196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:09.855196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:09.855196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:09.855196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:09.855196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:09.855196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:51:09.855196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:09.862660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 329/654] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:09.867335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d653e-bd72-4cd2-adc7-22d1cd77fd7b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:09.878764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:09.919839 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb45aa-1c77-477e-b714-ad7b1ecc1127 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:09.955039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171927dc-a30c-4ead-a709-1c744b1d93d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:09.955858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171927dc-a30c-4ead-a709-1c744b1d93d2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:09.956141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171927dc-a30c-4ead-a709-1c744b1d93d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:09.956419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171927dc-a30c-4ead-a709-1c744b1d93d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:09.967470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171927dc-a30c-4ead-a709-1c744b1d93d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:09.968664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d653e-bd72-4cd2-adc7-22d1cd77fd7b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:09.968958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171927dc-a30c-4ead-a709-1c744b1d93d2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-171927dc-a30c-4ead-a709-1c744b1d93d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:09.988097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:09.989038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:09.989905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:09.989905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:09.989905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:09.989905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:09.993012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 326/655] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:09.993887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171927dc-a30c-4ead-a709-1c744b1d93d2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:09.997441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:10.009657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6336ca-b45f-4ebd-96f2-f37afbcb74ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:10.010715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6336ca-b45f-4ebd-96f2-f37afbcb74ec None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:10.022070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6336ca-b45f-4ebd-96f2-f37afbcb74ec None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:10.022070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6336ca-b45f-4ebd-96f2-f37afbcb74ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:10.022561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6336ca-b45f-4ebd-96f2-f37afbcb74ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:10.024222 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6336ca-b45f-4ebd-96f2-f37afbcb74ec None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-9c6336ca-b45f-4ebd-96f2-f37afbcb74ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:10.042984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:10.043893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:10.044790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:10.044790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:10.044790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:10.044790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:10.051882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 330/656] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:10.053010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6336ca-b45f-4ebd-96f2-f37afbcb74ec None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:10.103399 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171927dc-a30c-4ead-a709-1c744b1d93d2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:10.121340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:10.142756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8cd61-c4e4-4704-89e4-62dd8b5f6f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:10.143459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8cd61-c4e4-4704-89e4-62dd8b5f6f95 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:10.143743 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8cd61-c4e4-4704-89e4-62dd8b5f6f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:10.144075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8cd61-c4e4-4704-89e4-62dd8b5f6f95 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:10.144951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8cd61-c4e4-4704-89e4-62dd8b5f6f95 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:10.145473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8cd61-c4e4-4704-89e4-62dd8b5f6f95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:10.146923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8cd61-c4e4-4704-89e4-62dd8b5f6f95 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:10.165073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6336ca-b45f-4ebd-96f2-f37afbcb74ec None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:10.171463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 327/657] 10.222.0.45 () {56 vars in 1137 bytes} [Tue Jul 5 15:51:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:10.171463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8cd61-c4e4-4704-89e4-62dd8b5f6f95 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:10.214660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c540d7-d80b-46f0-9427-830e79f41b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:10.215481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c540d7-d80b-46f0-9427-830e79f41b91 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:10.215871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c540d7-d80b-46f0-9427-830e79f41b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:10.216406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c540d7-d80b-46f0-9427-830e79f41b91 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:10.217270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c540d7-d80b-46f0-9427-830e79f41b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:10.218501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c540d7-d80b-46f0-9427-830e79f41b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:10.220058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c540d7-d80b-46f0-9427-830e79f41b91 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:10.236849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 331/658] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:10.237911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c540d7-d80b-46f0-9427-830e79f41b91 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:10.252176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:10.302303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8cd61-c4e4-4704-89e4-62dd8b5f6f95 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:10.337672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b837f8-98f2-4091-937c-845731c93a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:10.338550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b837f8-98f2-4091-937c-845731c93a46 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:10.338984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b837f8-98f2-4091-937c-845731c93a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:10.339437 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b837f8-98f2-4091-937c-845731c93a46 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:10.340303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b837f8-98f2-4091-937c-845731c93a46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:10.340759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b837f8-98f2-4091-937c-845731c93a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:10.342356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b837f8-98f2-4091-937c-845731c93a46 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:10.345032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c540d7-d80b-46f0-9427-830e79f41b91 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:10.360231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 328/659] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:10.361351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b837f8-98f2-4091-937c-845731c93a46 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:10.373135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:10.385723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee05fba-ee7e-4250-9583-1f7c80292e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:10.386475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05fba-ee7e-4250-9583-1f7c80292e9d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:10.386906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05fba-ee7e-4250-9583-1f7c80292e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:10.387246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee05fba-ee7e-4250-9583-1f7c80292e9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:10.461637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee05fba-ee7e-4250-9583-1f7c80292e9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:10.463415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee05fba-ee7e-4250-9583-1f7c80292e9d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:10.466178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b837f8-98f2-4091-937c-845731c93a46 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:10.488198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:10.503047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377d970-283b-4309-b3da-bbec30511f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:10.503940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377d970-283b-4309-b3da-bbec30511f0c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:10.504285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377d970-283b-4309-b3da-bbec30511f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:10.504622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377d970-283b-4309-b3da-bbec30511f0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:10.511975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377d970-283b-4309-b3da-bbec30511f0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:10.513579 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377d970-283b-4309-b3da-bbec30511f0c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3377d970-283b-4309-b3da-bbec30511f0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:10.528599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:10.529703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:10.530785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:10.530785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:10.530785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:10.530785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:10.530785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:10.530785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:10.530785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:10.530785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:10.537012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 329/660] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:10.537012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377d970-283b-4309-b3da-bbec30511f0c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:10.582858 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 332/661] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:51:10.584061 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee05fba-ee7e-4250-9583-1f7c80292e9d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:10.611223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:10.654061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377d970-283b-4309-b3da-bbec30511f0c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:10.695261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee05fba-ee7e-4250-9583-1f7c80292e9d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:10.699758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af678dd7-45fd-408f-98e1-64007169dd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:10.700566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af678dd7-45fd-408f-98e1-64007169dd1b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:10.700879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af678dd7-45fd-408f-98e1-64007169dd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:10.701160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af678dd7-45fd-408f-98e1-64007169dd1b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:10.710849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af678dd7-45fd-408f-98e1-64007169dd1b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:10.712964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af678dd7-45fd-408f-98e1-64007169dd1b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-af678dd7-45fd-408f-98e1-64007169dd1b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:10.731110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:10.732090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:10.733125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:10.733125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:10.733125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:10.733125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:10.733125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:10.733125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:10.733125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:10.733125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:10.733125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:51:10.733125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:10.735948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 330/662] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:10.737060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af678dd7-45fd-408f-98e1-64007169dd1b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:10.738650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd0337-d40e-4cc8-9700-687cd5f207b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:10.739317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd0337-d40e-4cc8-9700-687cd5f207b7 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:10.739607 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd0337-d40e-4cc8-9700-687cd5f207b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:10.739886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd0337-d40e-4cc8-9700-687cd5f207b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:10.740787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd0337-d40e-4cc8-9700-687cd5f207b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:10.741109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd0337-d40e-4cc8-9700-687cd5f207b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:10.743630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd0337-d40e-4cc8-9700-687cd5f207b7 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:10.763665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 333/663] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:10.763665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd0337-d40e-4cc8-9700-687cd5f207b7 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:10.790431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:10.859506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af678dd7-45fd-408f-98e1-64007169dd1b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:10.878110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:10.897233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd0337-d40e-4cc8-9700-687cd5f207b7 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:10.899137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974ea3-562a-4cb2-a931-fab986a84d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:10.900567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974ea3-562a-4cb2-a931-fab986a84d15 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:10.900567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974ea3-562a-4cb2-a931-fab986a84d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:10.900567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974ea3-562a-4cb2-a931-fab986a84d15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:10.911547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974ea3-562a-4cb2-a931-fab986a84d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:10.913041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974ea3-562a-4cb2-a931-fab986a84d15 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-33974ea3-562a-4cb2-a931-fab986a84d15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:10.931287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:10.932259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:10.933169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:10.933169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:10.933169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:10.933169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:10.933169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 331/664] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:10.933882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974ea3-562a-4cb2-a931-fab986a84d15 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:10.939668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212c80c-ebee-4666-b9a4-513da386c4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:10.941254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212c80c-ebee-4666-b9a4-513da386c4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:10.941759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212c80c-ebee-4666-b9a4-513da386c4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:10.942237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212c80c-ebee-4666-b9a4-513da386c4f8 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:10.943396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212c80c-ebee-4666-b9a4-513da386c4f8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:10.943911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212c80c-ebee-4666-b9a4-513da386c4f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:10.945589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212c80c-ebee-4666-b9a4-513da386c4f8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:10.966355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 334/665] 10.222.0.45 () {56 vars in 1139 bytes} [Tue Jul 5 15:51:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:10.967687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212c80c-ebee-4666-b9a4-513da386c4f8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:11.030927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:11.039701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974ea3-562a-4cb2-a931-fab986a84d15 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:11.077002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813da92-9560-420c-8883-fbc9186e01ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:11.077660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813da92-9560-420c-8883-fbc9186e01ec None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.077991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813da92-9560-420c-8883-fbc9186e01ec None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.078226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813da92-9560-420c-8883-fbc9186e01ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.079158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813da92-9560-420c-8883-fbc9186e01ec None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=805a9dd2d0934359b7ca5fe629b96d38) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:11.079476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813da92-9560-420c-8883-fbc9186e01ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:11.080261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212c80c-ebee-4666-b9a4-513da386c4f8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:11.080977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813da92-9560-420c-8883-fbc9186e01ec None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:11.088247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 332/666] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:10 2022] GET /identity/v3/role_assignments?user.id=805a9dd2d0934359b7ca5fe629b96d38&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:11.089232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813da92-9560-420c-8883-fbc9186e01ec None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:11.129270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f6aea-63ec-4595-b2f1-768113d3207e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:11.130054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f6aea-63ec-4595-b2f1-768113d3207e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.130413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f6aea-63ec-4595-b2f1-768113d3207e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.130765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f6aea-63ec-4595-b2f1-768113d3207e None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.131551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f6aea-63ec-4595-b2f1-768113d3207e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:11.131861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f6aea-63ec-4595-b2f1-768113d3207e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:11.133324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f6aea-63ec-4595-b2f1-768113d3207e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:11.145880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 335/667] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:11.151848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a445ce-01cf-4dbf-a64c-250a7c581ae7 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.152139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a445ce-01cf-4dbf-a64c-250a7c581ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.152424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a445ce-01cf-4dbf-a64c-250a7c581ae7 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.153825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 336/668] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:11.154956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8329a-2d7a-4a2a-a664-b76ffee3db0a None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.155238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8329a-2d7a-4a2a-a664-b76ffee3db0a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.155518 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8329a-2d7a-4a2a-a664-b76ffee3db0a None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.157050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 337/669] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:11.157584 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8329a-2d7a-4a2a-a664-b76ffee3db0a None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:11.160794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:11.217611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813da92-9560-420c-8883-fbc9186e01ec None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:11.261566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d59cf-edd2-44ce-a201-5878f7351f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:11.262221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d59cf-edd2-44ce-a201-5878f7351f81 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.262509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d59cf-edd2-44ce-a201-5878f7351f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.262790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d59cf-edd2-44ce-a201-5878f7351f81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.276231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8329a-2d7a-4a2a-a664-b76ffee3db0a None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:11.297748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:11.313850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c30cb8-238a-4b45-94f7-e167d032d4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:11.314646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c30cb8-238a-4b45-94f7-e167d032d4dc None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.315550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c30cb8-238a-4b45-94f7-e167d032d4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.315811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c30cb8-238a-4b45-94f7-e167d032d4dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.323079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c30cb8-238a-4b45-94f7-e167d032d4dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:11.324523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c30cb8-238a-4b45-94f7-e167d032d4dc None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-98c30cb8-238a-4b45-94f7-e167d032d4dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:11.338673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:11.339589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:11.340477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:11.340477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:11.340477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:11.340477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:11.340477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:11.340477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:11.340477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:11.340477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:11.340477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 338/670] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:11.341025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad560d-f2c2-43e8-b193-9e6eea4af6ef None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.341828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad560d-f2c2-43e8-b193-9e6eea4af6ef None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.341828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad560d-f2c2-43e8-b193-9e6eea4af6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.352211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d59cf-edd2-44ce-a201-5878f7351f81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:11.353709 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d59cf-edd2-44ce-a201-5878f7351f81 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:11.358957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-7bad560d-f2c2-43e8-b193-9e6eea4af6ef None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:11.360128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7bad560d-f2c2-43e8-b193-9e6eea4af6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:11.418787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:11.437896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 339/671] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:11.437896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddfc42-9713-451c-bcd9-488d6d9c5e22 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.437896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddfc42-9713-451c-bcd9-488d6d9c5e22 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.437896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddfc42-9713-451c-bcd9-488d6d9c5e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.457501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-04ddfc42-9713-451c-bcd9-488d6d9c5e22 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:11.458155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 333/672] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:51:11.458671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-04ddfc42-9713-451c-bcd9-488d6d9c5e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:11.459187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d59cf-edd2-44ce-a201-5878f7351f81 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:11.517364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:11.540791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 340/673] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:11.540791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddfc42-9713-451c-bcd9-488d6d9c5e22 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:11.565039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d59cf-edd2-44ce-a201-5878f7351f81 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:11.602166 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acd136-d978-4f8b-ac4e-aefd34d68069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:11.603067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acd136-d978-4f8b-ac4e-aefd34d68069 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.603353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acd136-d978-4f8b-ac4e-aefd34d68069 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.603635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acd136-d978-4f8b-ac4e-aefd34d68069 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.604426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acd136-d978-4f8b-ac4e-aefd34d68069 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:11.604737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acd136-d978-4f8b-ac4e-aefd34d68069 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:11.606200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acd136-d978-4f8b-ac4e-aefd34d68069 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:11.618640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 334/674] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:11.619434 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acd136-d978-4f8b-ac4e-aefd34d68069 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:11.655224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddfc42-9713-451c-bcd9-488d6d9c5e22 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:11.679549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:11.691886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bf361-bf14-4fc8-a6ce-6b3c9030dc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:11.692861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bf361-bf14-4fc8-a6ce-6b3c9030dc56 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.693090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bf361-bf14-4fc8-a6ce-6b3c9030dc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.693370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bf361-bf14-4fc8-a6ce-6b3c9030dc56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.704137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bf361-bf14-4fc8-a6ce-6b3c9030dc56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-c80bf361-bf14-4fc8-a6ce-6b3c9030dc56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:11.713650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:11.714576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:11.715401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:11.717891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 341/675] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:11.718810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bf361-bf14-4fc8-a6ce-6b3c9030dc56 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:11.727478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acd136-d978-4f8b-ac4e-aefd34d68069 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:11.747713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:11.770436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c1dcc-95e9-40c0-b044-2fee846dedc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:11.771502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c1dcc-95e9-40c0-b044-2fee846dedc5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.772066 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c1dcc-95e9-40c0-b044-2fee846dedc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.772574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c1dcc-95e9-40c0-b044-2fee846dedc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.787380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c1dcc-95e9-40c0-b044-2fee846dedc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:11.789872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c1dcc-95e9-40c0-b044-2fee846dedc5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5e0c1dcc-95e9-40c0-b044-2fee846dedc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:11.815189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:11.816755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:11.822710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:11.822710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:11.822710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:11.822710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:11.822710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 335/676] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:11.822710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c1dcc-95e9-40c0-b044-2fee846dedc5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:11.854062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bf361-bf14-4fc8-a6ce-6b3c9030dc56 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:11.903176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9d6e5-7637-4014-b070-e9fd836d4dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:11.903971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9d6e5-7637-4014-b070-e9fd836d4dae None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.904251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9d6e5-7637-4014-b070-e9fd836d4dae None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.904523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9d6e5-7637-4014-b070-e9fd836d4dae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.915800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9d6e5-7637-4014-b070-e9fd836d4dae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-41b9d6e5-7637-4014-b070-e9fd836d4dae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:11.923617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:11.924594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:11.925783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:11.929973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 342/677] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:11.930808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9d6e5-7637-4014-b070-e9fd836d4dae None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:11.946172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c1dcc-95e9-40c0-b044-2fee846dedc5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:11.977656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:11.992751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb99e7-50f2-476b-b58d-d9860c7975f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:11.999038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb99e7-50f2-476b-b58d-d9860c7975f4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:11.999038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb99e7-50f2-476b-b58d-d9860c7975f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:11.999038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb99e7-50f2-476b-b58d-d9860c7975f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:11.999038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb99e7-50f2-476b-b58d-d9860c7975f4 None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=a5b712bd7d8d4930a3bfc99581be8cb7) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:11.999038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb99e7-50f2-476b-b58d-d9860c7975f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:11.999038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb99e7-50f2-476b-b58d-d9860c7975f4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:12.004008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 336/678] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:11 2022] GET /identity/v3/role_assignments?user.id=a5b712bd7d8d4930a3bfc99581be8cb7&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:12.004806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb99e7-50f2-476b-b58d-d9860c7975f4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:12.063006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:12.083678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9d6e5-7637-4014-b070-e9fd836d4dae None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:12.174023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5753b4-a3d7-4405-b3c9-9f51941c2c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:12.174834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5753b4-a3d7-4405-b3c9-9f51941c2c61 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:12.175022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5753b4-a3d7-4405-b3c9-9f51941c2c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:12.175278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5753b4-a3d7-4405-b3c9-9f51941c2c61 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:12.176042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5753b4-a3d7-4405-b3c9-9f51941c2c61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:12.176350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5753b4-a3d7-4405-b3c9-9f51941c2c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:12.177780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5753b4-a3d7-4405-b3c9-9f51941c2c61 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:12.193352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 343/679] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:12.194048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5753b4-a3d7-4405-b3c9-9f51941c2c61 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:12.217908 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb99e7-50f2-476b-b58d-d9860c7975f4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:12.278804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2b4a1-0a79-456d-82d7-ecd976b7b1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:12.279477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2b4a1-0a79-456d-82d7-ecd976b7b1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:12.279764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2b4a1-0a79-456d-82d7-ecd976b7b1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:12.280043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2b4a1-0a79-456d-82d7-ecd976b7b1c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:12.280993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2b4a1-0a79-456d-82d7-ecd976b7b1c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:12.281146 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2b4a1-0a79-456d-82d7-ecd976b7b1c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:12.282658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2b4a1-0a79-456d-82d7-ecd976b7b1c5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:12.296707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 337/680] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:12.297867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2b4a1-0a79-456d-82d7-ecd976b7b1c5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:12.332101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5753b4-a3d7-4405-b3c9-9f51941c2c61 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:12.334970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:12.368112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd481771-ace2-4021-aa8a-b77c1de96f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:12.368796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd481771-ace2-4021-aa8a-b77c1de96f0c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:12.369098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd481771-ace2-4021-aa8a-b77c1de96f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:12.369379 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd481771-ace2-4021-aa8a-b77c1de96f0c None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:12.370132 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd481771-ace2-4021-aa8a-b77c1de96f0c None admin] RBAC: Adding query filter params (name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:12.370443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd481771-ace2-4021-aa8a-b77c1de96f0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:12.371931 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd481771-ace2-4021-aa8a-b77c1de96f0c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:12.385684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 344/681] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:12.385684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd481771-ace2-4021-aa8a-b77c1de96f0c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:12.409174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:12.422526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2b4a1-0a79-456d-82d7-ecd976b7b1c5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:12.466451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f0a23-f036-43a9-a7d7-8e29d59f6d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:12.468171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f0a23-f036-43a9-a7d7-8e29d59f6d18 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:12.468663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f0a23-f036-43a9-a7d7-8e29d59f6d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:12.469185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f0a23-f036-43a9-a7d7-8e29d59f6d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:12.487331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f0a23-f036-43a9-a7d7-8e29d59f6d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:12.490127 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f0a23-f036-43a9-a7d7-8e29d59f6d18 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:12.492685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd481771-ace2-4021-aa8a-b77c1de96f0c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-839f0a23-f036-43a9-a7d7-8e29d59f6d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:12.511933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:12.512906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:12.513786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:12.513786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:12.513786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:12.513786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:12.517752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 338/682] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:12.518606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f0a23-f036-43a9-a7d7-8e29d59f6d18 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:12.537955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926fb4f-bb93-4b89-94d6-b5c4e69e65ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:12.539068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926fb4f-bb93-4b89-94d6-b5c4e69e65ac None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:12.539552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926fb4f-bb93-4b89-94d6-b5c4e69e65ac None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:12.540027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926fb4f-bb93-4b89-94d6-b5c4e69e65ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:12.550499 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926fb4f-bb93-4b89-94d6-b5c4e69e65ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:12.552757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926fb4f-bb93-4b89-94d6-b5c4e69e65ac None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-8926fb4f-bb93-4b89-94d6-b5c4e69e65ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:12.572404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:12.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:12.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:12.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:12.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:12.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:12.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:12.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:12.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:12.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:12.577096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 345/683] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:12.578297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926fb4f-bb93-4b89-94d6-b5c4e69e65ac None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:12.617209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:12.626337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f0a23-f036-43a9-a7d7-8e29d59f6d18 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:12.664458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c5667-5a26-424a-822e-cff0c0e16843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:12.665495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c5667-5a26-424a-822e-cff0c0e16843 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:12.665973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c5667-5a26-424a-822e-cff0c0e16843 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:12.666381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c5667-5a26-424a-822e-cff0c0e16843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:12.678469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c5667-5a26-424a-822e-cff0c0e16843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:12.680215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c5667-5a26-424a-822e-cff0c0e16843 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:12.683078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-bc2c5667-5a26-424a-822e-cff0c0e16843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:12.701159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:12.702303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:12.703390 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:12.703390 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:12.703390 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:12.703390 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:12.709175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 339/684] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:12.709175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c5667-5a26-424a-822e-cff0c0e16843 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:12.714164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926fb4f-bb93-4b89-94d6-b5c4e69e65ac None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:12.757251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60252f63-93de-48c9-bd3d-f20b184003ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:12.758739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60252f63-93de-48c9-bd3d-f20b184003ef None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:12.759175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60252f63-93de-48c9-bd3d-f20b184003ef None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:12.759600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60252f63-93de-48c9-bd3d-f20b184003ef None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:12.760646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60252f63-93de-48c9-bd3d-f20b184003ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:12.761119 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60252f63-93de-48c9-bd3d-f20b184003ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:12.763318 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60252f63-93de-48c9-bd3d-f20b184003ef None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:12.778355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 346/685] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:12.779371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60252f63-93de-48c9-bd3d-f20b184003ef None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:12.828911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c5667-5a26-424a-822e-cff0c0e16843 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:12.872212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adabff-5b55-45f4-a511-6b8031d68c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:12.873070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adabff-5b55-45f4-a511-6b8031d68c5e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:12.873461 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adabff-5b55-45f4-a511-6b8031d68c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:12.874112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adabff-5b55-45f4-a511-6b8031d68c5e None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:12.874840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adabff-5b55-45f4-a511-6b8031d68c5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:12.875252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adabff-5b55-45f4-a511-6b8031d68c5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:12.877015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adabff-5b55-45f4-a511-6b8031d68c5e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:12.885810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:12.891729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 340/686] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:12.892809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adabff-5b55-45f4-a511-6b8031d68c5e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:12.907978 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60252f63-93de-48c9-bd3d-f20b184003ef None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:12.949789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabffcd5-0f68-4668-a47f-3b4fd8fb2d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:12.950782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffcd5-0f68-4668-a47f-3b4fd8fb2d1e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:12.950965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffcd5-0f68-4668-a47f-3b4fd8fb2d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:12.951347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffcd5-0f68-4668-a47f-3b4fd8fb2d1e None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:12.952177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabffcd5-0f68-4668-a47f-3b4fd8fb2d1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:12.952593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabffcd5-0f68-4668-a47f-3b4fd8fb2d1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:12.954085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabffcd5-0f68-4668-a47f-3b4fd8fb2d1e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:12.961656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:12.971568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 347/687] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:12.971568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabffcd5-0f68-4668-a47f-3b4fd8fb2d1e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:13.022778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adabff-5b55-45f4-a511-6b8031d68c5e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:13.063741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee19759-3bc1-43ee-9eb5-1159121c4f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:13.065248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19759-3bc1-43ee-9eb5-1159121c4f44 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:13.065636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19759-3bc1-43ee-9eb5-1159121c4f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:13.066015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19759-3bc1-43ee-9eb5-1159121c4f44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:13.075387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee19759-3bc1-43ee-9eb5-1159121c4f44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:13.077032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee19759-3bc1-43ee-9eb5-1159121c4f44 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-3ee19759-3bc1-43ee-9eb5-1159121c4f44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:13.092819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:13.093996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:13.095145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:13.095145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:13.095145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:13.095145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:13.095145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:13.095145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:13.095145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:13.095145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:13.095145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:13.095145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:13.102923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 341/688] 10.222.0.45 () {56 vars in 1093 bytes} [Tue Jul 5 15:51:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:13.103300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee19759-3bc1-43ee-9eb5-1159121c4f44 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:13.110200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabffcd5-0f68-4668-a47f-3b4fd8fb2d1e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:13.157303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcbd8d-6fd7-41f5-b58c-667df13e14e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:13.158589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcbd8d-6fd7-41f5-b58c-667df13e14e8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:13.159039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcbd8d-6fd7-41f5-b58c-667df13e14e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:13.159482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcbd8d-6fd7-41f5-b58c-667df13e14e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:13.160463 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcbd8d-6fd7-41f5-b58c-667df13e14e8 None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=887d2ad3cbf7459a8a59e7442ba9744e) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:13.161450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcbd8d-6fd7-41f5-b58c-667df13e14e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:13.163345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcbd8d-6fd7-41f5-b58c-667df13e14e8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:13.172624 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 348/689] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:12 2022] GET /identity/v3/role_assignments?user.id=887d2ad3cbf7459a8a59e7442ba9744e&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:13.173722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcbd8d-6fd7-41f5-b58c-667df13e14e8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:13.190286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:13.227163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee19759-3bc1-43ee-9eb5-1159121c4f44 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:13.250418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:13.279736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cadc5d-124a-4117-8674-ea6d84e1e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:13.280686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cadc5d-124a-4117-8674-ea6d84e1e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:13.281023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cadc5d-124a-4117-8674-ea6d84e1e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:13.284797 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cadc5d-124a-4117-8674-ea6d84e1e6c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:13.296624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cadc5d-124a-4117-8674-ea6d84e1e6c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:13.297525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cadc5d-124a-4117-8674-ea6d84e1e6c7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-54cadc5d-124a-4117-8674-ea6d84e1e6c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:13.315306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:13.316523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:13.317565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 342/690] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:13.318369 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efd4da-4f1a-4cb6-9a06-d479a63fd78a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:13.318666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efd4da-4f1a-4cb6-9a06-d479a63fd78a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:13.318948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efd4da-4f1a-4cb6-9a06-d479a63fd78a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:13.323756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcbd8d-6fd7-41f5-b58c-667df13e14e8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:13.324551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 343/691] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:13.325683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efd4da-4f1a-4cb6-9a06-d479a63fd78a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:13.374901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6717ace-8207-4d1c-a016-a1c662a9f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:13.375112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6717ace-8207-4d1c-a016-a1c662a9f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:13.376619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6717ace-8207-4d1c-a016-a1c662a9f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:13.376619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6717ace-8207-4d1c-a016-a1c662a9f5e8 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:13.376619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6717ace-8207-4d1c-a016-a1c662a9f5e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:13.376885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6717ace-8207-4d1c-a016-a1c662a9f5e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:13.379435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6717ace-8207-4d1c-a016-a1c662a9f5e8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:13.399444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 349/692] 10.222.0.45 () {56 vars in 1133 bytes} [Tue Jul 5 15:51:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:13.400945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608bdc5-cecb-40c6-a672-9e6d173d3b39 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:13.401242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608bdc5-cecb-40c6-a672-9e6d173d3b39 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:13.401522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608bdc5-cecb-40c6-a672-9e6d173d3b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:13.418426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f608bdc5-cecb-40c6-a672-9e6d173d3b39 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:13.419630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f608bdc5-cecb-40c6-a672-9e6d173d3b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:13.451142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efd4da-4f1a-4cb6-9a06-d479a63fd78a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:13.493883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6415c1-6760-4ed5-a450-faffb6bf5da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:13.494556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6415c1-6760-4ed5-a450-faffb6bf5da1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:13.494842 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6415c1-6760-4ed5-a450-faffb6bf5da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:13.495119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6415c1-6760-4ed5-a450-faffb6bf5da1 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:13.495729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:13.495960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6415c1-6760-4ed5-a450-faffb6bf5da1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:13.496284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6415c1-6760-4ed5-a450-faffb6bf5da1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:13.497840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6415c1-6760-4ed5-a450-faffb6bf5da1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:13.505984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 350/693] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:13.507083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608bdc5-cecb-40c6-a672-9e6d173d3b39 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:13.533380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 344/694] 10.222.0.45 () {56 vars in 1135 bytes} [Tue Jul 5 15:51:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:13.542405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6415c1-6760-4ed5-a450-faffb6bf5da1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:13.579642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:13.636694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608bdc5-cecb-40c6-a672-9e6d173d3b39 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:13.653252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6415c1-6760-4ed5-a450-faffb6bf5da1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:13.673693 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707580d-bed7-4f75-b430-9e46ca9c9ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:13.675140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707580d-bed7-4f75-b430-9e46ca9c9ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:13.675570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707580d-bed7-4f75-b430-9e46ca9c9ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:13.676104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707580d-bed7-4f75-b430-9e46ca9c9ce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:13.691038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613ebcc-1900-4e92-b2c0-f60d96f14934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:13.691691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613ebcc-1900-4e92-b2c0-f60d96f14934 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:13.692036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613ebcc-1900-4e92-b2c0-f60d96f14934 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:13.692451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613ebcc-1900-4e92-b2c0-f60d96f14934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:13.747583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:13.751551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707580d-bed7-4f75-b430-9e46ca9c9ce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:13.753381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707580d-bed7-4f75-b430-9e46ca9c9ce4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:13.779153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613ebcc-1900-4e92-b2c0-f60d96f14934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:13.780794 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613ebcc-1900-4e92-b2c0-f60d96f14934 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:13.821939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:13.864281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 351/695] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:51:13.865560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707580d-bed7-4f75-b430-9e46ca9c9ce4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:13.903197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 345/696] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:51:13.903197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613ebcc-1900-4e92-b2c0-f60d96f14934 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:13.978980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:13.983240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707580d-bed7-4f75-b430-9e46ca9c9ce4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:14.022612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb1609-bf72-4c06-b0f4-27996190bdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:14.023681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb1609-bf72-4c06-b0f4-27996190bdef None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.024115 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb1609-bf72-4c06-b0f4-27996190bdef None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.024536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb1609-bf72-4c06-b0f4-27996190bdef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.025368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb1609-bf72-4c06-b0f4-27996190bdef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:14.027044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb1609-bf72-4c06-b0f4-27996190bdef None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-5abb1609-bf72-4c06-b0f4-27996190bdef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:14.043216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:14.044445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:51:14.045425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:14.052450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 352/697] 10.222.0.45 () {56 vars in 1129 bytes} [Tue Jul 5 15:51:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:14.053473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb1609-bf72-4c06-b0f4-27996190bdef None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:14.134112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613ebcc-1900-4e92-b2c0-f60d96f14934 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:14.165707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb1609-bf72-4c06-b0f4-27996190bdef None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:14.172024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b8aa7-4484-48de-bdd9-8b82b99b5900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:14.172838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b8aa7-4484-48de-bdd9-8b82b99b5900 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.173126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b8aa7-4484-48de-bdd9-8b82b99b5900 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.173449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b8aa7-4484-48de-bdd9-8b82b99b5900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.175625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:14.185478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b8aa7-4484-48de-bdd9-8b82b99b5900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:14.187052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b8aa7-4484-48de-bdd9-8b82b99b5900 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:14.203733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b958b-1880-4bde-80ce-fcb868dd02c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:14.205145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b958b-1880-4bde-80ce-fcb868dd02c2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.206141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b958b-1880-4bde-80ce-fcb868dd02c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8e2b8aa7-4484-48de-bdd9-8b82b99b5900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:14.206404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:14.207431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:14.208391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:14.208391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:14.208391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:14.208391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:14.208391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 346/698] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:14.208826 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b8aa7-4484-48de-bdd9-8b82b99b5900 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:14.209059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b958b-1880-4bde-80ce-fcb868dd02c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.224457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b958b-1880-4bde-80ce-fcb868dd02c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:14.227302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b958b-1880-4bde-80ce-fcb868dd02c2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-da7b958b-1880-4bde-80ce-fcb868dd02c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:14.246267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:14.247364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:14.248332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:14.248332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:14.248332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:14.248332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:14.254761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 353/699] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:14.255936 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b958b-1880-4bde-80ce-fcb868dd02c2 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:14.273584 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:14.322849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b8aa7-4484-48de-bdd9-8b82b99b5900 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:14.363901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b958b-1880-4bde-80ce-fcb868dd02c2 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:14.364084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade2e7a-0ca7-4f35-9ccf-7afaab59d2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:14.364721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade2e7a-0ca7-4f35-9ccf-7afaab59d2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.365039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade2e7a-0ca7-4f35-9ccf-7afaab59d2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.365443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade2e7a-0ca7-4f35-9ccf-7afaab59d2b4 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.366031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade2e7a-0ca7-4f35-9ccf-7afaab59d2b4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:14.366348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade2e7a-0ca7-4f35-9ccf-7afaab59d2b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:14.367729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade2e7a-0ca7-4f35-9ccf-7afaab59d2b4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:14.372398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 347/700] 10.222.0.45 () {56 vars in 1124 bytes} [Tue Jul 5 15:51:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:14.373206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade2e7a-0ca7-4f35-9ccf-7afaab59d2b4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:14.414911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbf77c-afec-4436-90bf-75922181475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:14.415931 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbf77c-afec-4436-90bf-75922181475e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.416840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbf77c-afec-4436-90bf-75922181475e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.417328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbf77c-afec-4436-90bf-75922181475e None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.418239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbf77c-afec-4436-90bf-75922181475e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:14.418721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbf77c-afec-4436-90bf-75922181475e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:14.420285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbf77c-afec-4436-90bf-75922181475e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:14.427635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:14.438046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 354/701] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:14.438046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97841bc7-16ff-490a-8837-2a1331a15abb None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.438046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97841bc7-16ff-490a-8837-2a1331a15abb None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.438046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97841bc7-16ff-490a-8837-2a1331a15abb None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.438046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 355/702] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:14.438046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97841bc7-16ff-490a-8837-2a1331a15abb None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:14.490564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade2e7a-0ca7-4f35-9ccf-7afaab59d2b4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:14.520682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:14.525623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce26ff-687b-445d-847c-ade4d26905d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:14.526339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce26ff-687b-445d-847c-ade4d26905d2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.526562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce26ff-687b-445d-847c-ade4d26905d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.532346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce26ff-687b-445d-847c-ade4d26905d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.532346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce26ff-687b-445d-847c-ade4d26905d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:14.532346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce26ff-687b-445d-847c-ade4d26905d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:14.532346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce26ff-687b-445d-847c-ade4d26905d2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:14.545030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 348/703] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:14.545883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce26ff-687b-445d-847c-ade4d26905d2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:14.552383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97841bc7-16ff-490a-8837-2a1331a15abb None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:14.587540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce2916-aa3e-4d4f-b526-085d2295985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:14.588304 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce2916-aa3e-4d4f-b526-085d2295985a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.588590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce2916-aa3e-4d4f-b526-085d2295985a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.588891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce2916-aa3e-4d4f-b526-085d2295985a None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.589590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce2916-aa3e-4d4f-b526-085d2295985a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:14.589891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce2916-aa3e-4d4f-b526-085d2295985a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:14.591358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce2916-aa3e-4d4f-b526-085d2295985a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:14.604674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 356/704] 10.222.0.45 () {56 vars in 1124 bytes} [Tue Jul 5 15:51:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:14.604674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c06091-44cf-4a60-942d-b4f696172436 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.604674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c06091-44cf-4a60-942d-b4f696172436 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.604674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c06091-44cf-4a60-942d-b4f696172436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.621540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b3c06091-44cf-4a60-942d-b4f696172436 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:14.622917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b3c06091-44cf-4a60-942d-b4f696172436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:14.667925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce26ff-687b-445d-847c-ade4d26905d2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:14.682502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:14.710209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d9a96-a305-401b-8327-6e9d85518d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:14.711322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 357/705] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:14.711817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d9a96-a305-401b-8327-6e9d85518d3f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.712486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d9a96-a305-401b-8327-6e9d85518d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.712821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c06091-44cf-4a60-942d-b4f696172436 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:14.713599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d9a96-a305-401b-8327-6e9d85518d3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.722772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d9a96-a305-401b-8327-6e9d85518d3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:14.724595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d9a96-a305-401b-8327-6e9d85518d3f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-bc6d9a96-a305-401b-8327-6e9d85518d3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:14.741211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:14.742172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:14.743057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:14.743057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:14.743057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:14.743057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:14.743057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:14.743057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:14.743057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:14.743057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:14.750325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 349/706] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:14.752354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d9a96-a305-401b-8327-6e9d85518d3f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:14.797877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:14.847966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c06091-44cf-4a60-942d-b4f696172436 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:14.890786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d9a96-a305-401b-8327-6e9d85518d3f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:14.891744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e87351-b243-4ec4-b755-aaa2db2da0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:14.892554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e87351-b243-4ec4-b755-aaa2db2da0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.893000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e87351-b243-4ec4-b755-aaa2db2da0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.893275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e87351-b243-4ec4-b755-aaa2db2da0f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.901974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e87351-b243-4ec4-b755-aaa2db2da0f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:14.903912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e87351-b243-4ec4-b755-aaa2db2da0f0 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:14.936097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4967e52-3eb4-415e-a641-c72c716f6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:14.936782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4967e52-3eb4-415e-a641-c72c716f6fba None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:14.937088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4967e52-3eb4-415e-a641-c72c716f6fba None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:14.937364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4967e52-3eb4-415e-a641-c72c716f6fba None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:14.938168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4967e52-3eb4-415e-a641-c72c716f6fba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:14.938482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4967e52-3eb4-415e-a641-c72c716f6fba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:14.940553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4967e52-3eb4-415e-a641-c72c716f6fba None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:14.959560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 350/707] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:14.959560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4967e52-3eb4-415e-a641-c72c716f6fba None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:14.990713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-89e87351-b243-4ec4-b755-aaa2db2da0f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:15.050420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:15.051406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:15.052335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:15.052335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:15.052335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:15.052335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:15.052335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:15.052335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:15.052335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:15.052335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:15.060169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 358/708] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:15.061292 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e87351-b243-4ec4-b755-aaa2db2da0f0 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:15.068318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4967e52-3eb4-415e-a641-c72c716f6fba None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:15.105464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5df691-8e16-4014-8510-0eef3f052f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:15.106262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5df691-8e16-4014-8510-0eef3f052f98 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:15.106547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5df691-8e16-4014-8510-0eef3f052f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:15.106824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5df691-8e16-4014-8510-0eef3f052f98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:15.119805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5df691-8e16-4014-8510-0eef3f052f98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:15.121312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5df691-8e16-4014-8510-0eef3f052f98 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-bc5df691-8e16-4014-8510-0eef3f052f98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:15.139986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:15.141360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:15.143053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:15.143053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:15.143053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:15.143053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:15.143053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 351/709] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:15.143053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5df691-8e16-4014-8510-0eef3f052f98 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:15.179040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e87351-b243-4ec4-b755-aaa2db2da0f0 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:15.207391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:15.216805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f48c8-a407-4974-9ea5-8cbeace033ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:15.217803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f48c8-a407-4974-9ea5-8cbeace033ae None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:15.224742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f48c8-a407-4974-9ea5-8cbeace033ae None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:15.224742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f48c8-a407-4974-9ea5-8cbeace033ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:15.224742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f48c8-a407-4974-9ea5-8cbeace033ae None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=ef95ab7b924d456195a8761e894d911b) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:15.224742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f48c8-a407-4974-9ea5-8cbeace033ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:15.224742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f48c8-a407-4974-9ea5-8cbeace033ae None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:15.235089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:15.235454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 359/710] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:15 2022] GET /identity/v3/role_assignments?user.id=ef95ab7b924d456195a8761e894d911b&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:15.237473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f48c8-a407-4974-9ea5-8cbeace033ae None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:15.256412 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5df691-8e16-4014-8510-0eef3f052f98 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:15.294505 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f406c9-af9a-4226-a135-8af3dcc24014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:15.295282 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f406c9-af9a-4226-a135-8af3dcc24014 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:15.296007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f406c9-af9a-4226-a135-8af3dcc24014 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:15.296007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f406c9-af9a-4226-a135-8af3dcc24014 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:15.296981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f406c9-af9a-4226-a135-8af3dcc24014 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:15.297404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f406c9-af9a-4226-a135-8af3dcc24014 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:15.304903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f406c9-af9a-4226-a135-8af3dcc24014 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:15.313327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 352/711] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:15.320012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f406c9-af9a-4226-a135-8af3dcc24014 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:15.425523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f48c8-a407-4974-9ea5-8cbeace033ae None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:15.452914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f406c9-af9a-4226-a135-8af3dcc24014 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:15.469851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d3d67-2497-4888-b7f3-ba11dbaf97ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:15.470575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d3d67-2497-4888-b7f3-ba11dbaf97ad None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:15.473197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d3d67-2497-4888-b7f3-ba11dbaf97ad None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:15.473197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d3d67-2497-4888-b7f3-ba11dbaf97ad None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:15.473197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d3d67-2497-4888-b7f3-ba11dbaf97ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:15.473197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d3d67-2497-4888-b7f3-ba11dbaf97ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:15.474162 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d3d67-2497-4888-b7f3-ba11dbaf97ad None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:15.497648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 360/712] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:15.497648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7d1d9-db15-460e-9b03-4b79a1cb8fdd None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:15.497648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7d1d9-db15-460e-9b03-4b79a1cb8fdd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:15.497648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7d1d9-db15-460e-9b03-4b79a1cb8fdd None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:15.500360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b29602-88dd-4f78-a3b0-8056cd3736df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:15.501171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 361/713] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:15.501530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b29602-88dd-4f78-a3b0-8056cd3736df None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:15.501950 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7d1d9-db15-460e-9b03-4b79a1cb8fdd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:15.502455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b29602-88dd-4f78-a3b0-8056cd3736df None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:15.502916 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b29602-88dd-4f78-a3b0-8056cd3736df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:15.504131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b29602-88dd-4f78-a3b0-8056cd3736df None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=8da9cdd70af646928ee0edcadbcb524f) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:15.504704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b29602-88dd-4f78-a3b0-8056cd3736df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:15.506284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b29602-88dd-4f78-a3b0-8056cd3736df None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:15.513327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 353/714] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:15 2022] GET /identity/v3/role_assignments?user.id=8da9cdd70af646928ee0edcadbcb524f&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:15.514577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e117b2-613f-4a86-b3bf-130d8d479969 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:15.514867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e117b2-613f-4a86-b3bf-130d8d479969 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:15.515149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e117b2-613f-4a86-b3bf-130d8d479969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:15.539601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:15.553969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d7e117b2-613f-4a86-b3bf-130d8d479969 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:15.555275 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d7e117b2-613f-4a86-b3bf-130d8d479969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:15.625330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:15.693233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7d1d9-db15-460e-9b03-4b79a1cb8fdd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:15.711747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 354/715] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:15.741323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e117b2-613f-4a86-b3bf-130d8d479969 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:15.746319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee444ec4-2ec0-47d6-a5c8-74fc7a143d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:15.747004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee444ec4-2ec0-47d6-a5c8-74fc7a143d2d None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:15.747289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee444ec4-2ec0-47d6-a5c8-74fc7a143d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:15.747569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee444ec4-2ec0-47d6-a5c8-74fc7a143d2d None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:15.748285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee444ec4-2ec0-47d6-a5c8-74fc7a143d2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:15.749751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee444ec4-2ec0-47d6-a5c8-74fc7a143d2d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:15.836098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 362/716] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:51:15 2022] POST /identity/v3/users => generated 254 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:15.880150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e117b2-613f-4a86-b3bf-130d8d479969 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:15.921144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94f9e7-0656-48ac-8ae1-5d986ba6c774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:15.921922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94f9e7-0656-48ac-8ae1-5d986ba6c774 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:15.922200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94f9e7-0656-48ac-8ae1-5d986ba6c774 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:15.922475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94f9e7-0656-48ac-8ae1-5d986ba6c774 None admin] PATH_INFO: `/v3/services/network` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:15.923249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94f9e7-0656-48ac-8ae1-5d986ba6c774 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:15.924688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94f9e7-0656-48ac-8ae1-5d986ba6c774 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:15.928895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-9a94f9e7-0656-48ac-8ae1-5d986ba6c774 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:15.941228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:15.942331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:15.943185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 355/717] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:15 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:15.950042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee444ec4-2ec0-47d6-a5c8-74fc7a143d2d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:16.070164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee444ec4-2ec0-47d6-a5c8-74fc7a143d2d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:16.079484 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:16.111381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5249a-1802-426c-87be-4f3a6b757f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:16.112017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5249a-1802-426c-87be-4f3a6b757f35 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:16.112304 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5249a-1802-426c-87be-4f3a6b757f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:16.112580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5249a-1802-426c-87be-4f3a6b757f35 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:16.113324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5249a-1802-426c-87be-4f3a6b757f35 None admin] RBAC: Adding query filter params (name=network) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:16.113633 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5249a-1802-426c-87be-4f3a6b757f35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:16.114998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5249a-1802-426c-87be-4f3a6b757f35 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:16.124719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 363/718] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:16.134340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94f9e7-0656-48ac-8ae1-5d986ba6c774 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:16.258883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94f9e7-0656-48ac-8ae1-5d986ba6c774 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:16.305934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895b3a2-bf97-4fea-a5df-3da8a38dc778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:16.306606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895b3a2-bf97-4fea-a5df-3da8a38dc778 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:16.306890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895b3a2-bf97-4fea-a5df-3da8a38dc778 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:16.307165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895b3a2-bf97-4fea-a5df-3da8a38dc778 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:16.307880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895b3a2-bf97-4fea-a5df-3da8a38dc778 None admin] RBAC: Adding query filter params (type=network) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:16.308187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895b3a2-bf97-4fea-a5df-3da8a38dc778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:16.309601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895b3a2-bf97-4fea-a5df-3da8a38dc778 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:16.317154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 356/719] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:16.841707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80715b-d9e7-44ea-b0fe-8702ca1c722e None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:16.842376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80715b-d9e7-44ea-b0fe-8702ca1c722e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:16.842376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80715b-d9e7-44ea-b0fe-8702ca1c722e None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:16.843967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 364/720] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:16.850050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eacfa1-79c2-4188-918a-e774404d1db1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:16.850560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eacfa1-79c2-4188-918a-e774404d1db1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:16.850976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eacfa1-79c2-4188-918a-e774404d1db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:16.876604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-85eacfa1-79c2-4188-918a-e774404d1db1 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:16.877832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-85eacfa1-79c2-4188-918a-e774404d1db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:16.960611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:16.972819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 357/721] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:17.007373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80715b-d9e7-44ea-b0fe-8702ca1c722e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:17.145987 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80715b-d9e7-44ea-b0fe-8702ca1c722e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:17.197057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ca3dc-3590-4813-a702-4ad1d365ebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:17.197723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ca3dc-3590-4813-a702-4ad1d365ebed None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:17.198003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ca3dc-3590-4813-a702-4ad1d365ebed None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:17.198273 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ca3dc-3590-4813-a702-4ad1d365ebed None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:17.199000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ca3dc-3590-4813-a702-4ad1d365ebed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:17.200492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ca3dc-3590-4813-a702-4ad1d365ebed None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:17.302514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 365/722] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:51:17 2022] POST /identity/v3/services => generated 251 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:17.593361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c735c-4e68-441d-843d-082a10c3b043 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:17.593898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c735c-4e68-441d-843d-082a10c3b043 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:17.593898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c735c-4e68-441d-843d-082a10c3b043 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:17.595389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 358/723] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:17.601265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecefdb-3e3b-4719-b8f1-4fdf0c6ab0ba None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:17.601555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecefdb-3e3b-4719-b8f1-4fdf0c6ab0ba None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:17.602089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecefdb-3e3b-4719-b8f1-4fdf0c6ab0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:17.619067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-49ecefdb-3e3b-4719-b8f1-4fdf0c6ab0ba None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:17.620247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-49ecefdb-3e3b-4719-b8f1-4fdf0c6ab0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:17.648406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:17.708837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 366/724] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:17.731842 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c735c-4e68-441d-843d-082a10c3b043 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:17.869537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c735c-4e68-441d-843d-082a10c3b043 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:17.911492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058ea3a-ca70-4c4a-8b45-7b2c4fb70bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:17.912343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058ea3a-ca70-4c4a-8b45-7b2c4fb70bdd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:17.912627 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058ea3a-ca70-4c4a-8b45-7b2c4fb70bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:17.912924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058ea3a-ca70-4c4a-8b45-7b2c4fb70bdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:17.933960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058ea3a-ca70-4c4a-8b45-7b2c4fb70bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c058ea3a-ca70-4c4a-8b45-7b2c4fb70bdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:17.942196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:17.943136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:17.943966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 359/725] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:17.951290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecefdb-3e3b-4719-b8f1-4fdf0c6ab0ba None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:18.113023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecefdb-3e3b-4719-b8f1-4fdf0c6ab0ba None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:18.155670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece565f-60d7-492d-8d34-e8f40508aba8 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:18.155670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece565f-60d7-492d-8d34-e8f40508aba8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:18.155670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece565f-60d7-492d-8d34-e8f40508aba8 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:18.160596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 360/726] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:18.160753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9918d6a-b9ab-4d8c-bc5b-6815745cb052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:18.160753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9918d6a-b9ab-4d8c-bc5b-6815745cb052 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:18.160753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9918d6a-b9ab-4d8c-bc5b-6815745cb052 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:18.160753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9918d6a-b9ab-4d8c-bc5b-6815745cb052 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:18.164658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9918d6a-b9ab-4d8c-bc5b-6815745cb052 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:18.164896 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3848-da83-4c86-a732-e64b57ea865c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:18.164896 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3848-da83-4c86-a732-e64b57ea865c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:18.165194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9918d6a-b9ab-4d8c-bc5b-6815745cb052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:18.165359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3848-da83-4c86-a732-e64b57ea865c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:18.165521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9918d6a-b9ab-4d8c-bc5b-6815745cb052 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:18.173891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 367/727] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:18.180083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9918d6a-b9ab-4d8c-bc5b-6815745cb052 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:18.202141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:18.205224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b21a3848-da83-4c86-a732-e64b57ea865c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:18.206436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b21a3848-da83-4c86-a732-e64b57ea865c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:18.317115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:18.337671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9918d6a-b9ab-4d8c-bc5b-6815745cb052 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:18.353292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 361/728] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:51:18.380739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a3848-da83-4c86-a732-e64b57ea865c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:18.393884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb5bae-c5eb-4952-bdd2-4f57d3d2a789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:18.394807 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb5bae-c5eb-4952-bdd2-4f57d3d2a789 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:18.395095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb5bae-c5eb-4952-bdd2-4f57d3d2a789 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:18.395376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb5bae-c5eb-4952-bdd2-4f57d3d2a789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:18.418186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb5bae-c5eb-4952-bdd2-4f57d3d2a789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:18.419644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb5bae-c5eb-4952-bdd2-4f57d3d2a789 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-2bdb5bae-c5eb-4952-bdd2-4f57d3d2a789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:18.445055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:18.445988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:18.447294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:18.447294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:18.447294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:18.447294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:18.447294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 368/729] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:18.452948 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb5bae-c5eb-4952-bdd2-4f57d3d2a789 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:18.566442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:18.586595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb5bae-c5eb-4952-bdd2-4f57d3d2a789 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:18.624160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d968ec-39f3-405e-a791-7666c5318d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:18.624856 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d968ec-39f3-405e-a791-7666c5318d93 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:18.625145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d968ec-39f3-405e-a791-7666c5318d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:18.625418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d968ec-39f3-405e-a791-7666c5318d93 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:18.626143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d968ec-39f3-405e-a791-7666c5318d93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:18.626448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d968ec-39f3-405e-a791-7666c5318d93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:18.627836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d968ec-39f3-405e-a791-7666c5318d93 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:18.640617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 369/730] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:18.646664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d968ec-39f3-405e-a791-7666c5318d93 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:18.670805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a3848-da83-4c86-a732-e64b57ea865c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:18.752330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d968ec-39f3-405e-a791-7666c5318d93 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:18.786238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb93761-c2fa-4e93-af25-f865bfaa68b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:18.787304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb93761-c2fa-4e93-af25-f865bfaa68b1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:18.787585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb93761-c2fa-4e93-af25-f865bfaa68b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:18.787850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb93761-c2fa-4e93-af25-f865bfaa68b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:18.787951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:18.791709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e0757-cde2-4b7b-9d94-ff1ebb1b37a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:18.792745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e0757-cde2-4b7b-9d94-ff1ebb1b37a8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:18.793486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e0757-cde2-4b7b-9d94-ff1ebb1b37a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:18.793943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e0757-cde2-4b7b-9d94-ff1ebb1b37a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:18.799596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb93761-c2fa-4e93-af25-f865bfaa68b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:18.803224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e0757-cde2-4b7b-9d94-ff1ebb1b37a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:18.804886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e0757-cde2-4b7b-9d94-ff1ebb1b37a8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1fb93761-c2fa-4e93-af25-f865bfaa68b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:18.808597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:18.809542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:18.810391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 362/731] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:18.811898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbe7a2-cccf-4f64-ac83-4044798b238b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:18.812287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbe7a2-cccf-4f64-ac83-4044798b238b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:18.812660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbe7a2-cccf-4f64-ac83-4044798b238b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:18.816846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 363/732] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:18.817782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbe7a2-cccf-4f64-ac83-4044798b238b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-546e0757-cde2-4b7b-9d94-ff1ebb1b37a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:18.833025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:18.834235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:18.835524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:18.835524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:18.835524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:18.835524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:18.835524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:18.835524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:18.835524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:18.835524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:18.835524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:18.835524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:18.838237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 370/733] 10.222.0.45 () {56 vars in 1093 bytes} [Tue Jul 5 15:51:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:18.843904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e66cc-fefd-4196-9284-d2c1b3cb3f95 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:18.843904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e66cc-fefd-4196-9284-d2c1b3cb3f95 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:18.843904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e66cc-fefd-4196-9284-d2c1b3cb3f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:18.863368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4d8e66cc-fefd-4196-9284-d2c1b3cb3f95 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:18.865508 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4d8e66cc-fefd-4196-9284-d2c1b3cb3f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:18.882994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:18.954195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbe7a2-cccf-4f64-ac83-4044798b238b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:18.956837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 371/734] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:18.958289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e66cc-fefd-4196-9284-d2c1b3cb3f95 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:18.999056 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694f27b-f143-4784-827a-d4c828386d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:18.999726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694f27b-f143-4784-827a-d4c828386d95 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.000011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694f27b-f143-4784-827a-d4c828386d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.000289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694f27b-f143-4784-827a-d4c828386d95 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.001072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694f27b-f143-4784-827a-d4c828386d95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:19.001385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694f27b-f143-4784-827a-d4c828386d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:19.003292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694f27b-f143-4784-827a-d4c828386d95 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:19.011428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 364/735] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:19.012623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caee3c5-a2ce-48db-bab2-c83cd562c941 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.012941 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caee3c5-a2ce-48db-bab2-c83cd562c941 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.013218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caee3c5-a2ce-48db-bab2-c83cd562c941 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.014578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 365/736] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:19.015113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caee3c5-a2ce-48db-bab2-c83cd562c941 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:19.074356 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e66cc-fefd-4196-9284-d2c1b3cb3f95 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:19.097961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:19.109243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dda978-dbf3-4db4-bdc9-8c4a0b8f02e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:19.120634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda978-dbf3-4db4-bdc9-8c4a0b8f02e1 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.120634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda978-dbf3-4db4-bdc9-8c4a0b8f02e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.120634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dda978-dbf3-4db4-bdc9-8c4a0b8f02e1 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.120634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dda978-dbf3-4db4-bdc9-8c4a0b8f02e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:19.120634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dda978-dbf3-4db4-bdc9-8c4a0b8f02e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:19.120634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dda978-dbf3-4db4-bdc9-8c4a0b8f02e1 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:19.121183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caee3c5-a2ce-48db-bab2-c83cd562c941 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:19.128327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:19.135921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 372/737] 10.222.0.45 () {56 vars in 1133 bytes} [Tue Jul 5 15:51:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:19.135921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dda978-dbf3-4db4-bdc9-8c4a0b8f02e1 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:19.159407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2115b0-e956-4ee2-8ab8-161caa8ae952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:19.160190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2115b0-e956-4ee2-8ab8-161caa8ae952 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.160472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2115b0-e956-4ee2-8ab8-161caa8ae952 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.160746 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2115b0-e956-4ee2-8ab8-161caa8ae952 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.171595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2115b0-e956-4ee2-8ab8-161caa8ae952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ec2115b0-e956-4ee2-8ab8-161caa8ae952 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:19.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:19.180194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:19.181419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 366/738] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:19.182149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8466d5-935c-45a5-b60d-9d0fd9b1278d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.182149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8466d5-935c-45a5-b60d-9d0fd9b1278d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.182269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8466d5-935c-45a5-b60d-9d0fd9b1278d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.199537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2f8466d5-935c-45a5-b60d-9d0fd9b1278d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:19.200712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2f8466d5-935c-45a5-b60d-9d0fd9b1278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:19.244669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dda978-dbf3-4db4-bdc9-8c4a0b8f02e1 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:19.276708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 367/739] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:51:19.281411 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c334-5ab6-406f-a947-f5ff3a76f5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:19.281906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8466d5-935c-45a5-b60d-9d0fd9b1278d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:19.282204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c334-5ab6-406f-a947-f5ff3a76f5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.282488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c334-5ab6-406f-a947-f5ff3a76f5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.282818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c334-5ab6-406f-a947-f5ff3a76f5c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.294246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690c334-5ab6-406f-a947-f5ff3a76f5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:19.300854 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690c334-5ab6-406f-a947-f5ff3a76f5c6 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-1690c334-5ab6-406f-a947-f5ff3a76f5c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:19.313791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:19.315249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:19.316147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:19.316147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:19.316147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:19.316147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:19.316147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 373/740] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:19.316434 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c334-5ab6-406f-a947-f5ff3a76f5c6 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:19.352337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:19.388140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:19.395991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8466d5-935c-45a5-b60d-9d0fd9b1278d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:19.425106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c334-5ab6-406f-a947-f5ff3a76f5c6 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:19.431435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0568c9-0721-4382-bf59-93ae1a0bcbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:19.432224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0568c9-0721-4382-bf59-93ae1a0bcbdb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.432515 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0568c9-0721-4382-bf59-93ae1a0bcbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.432806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0568c9-0721-4382-bf59-93ae1a0bcbdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.444194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0568c9-0721-4382-bf59-93ae1a0bcbdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:19.445687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0568c9-0721-4382-bf59-93ae1a0bcbdb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:19.461663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2424d-27ca-4dd8-8f41-fce9c3fa53e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:19.462327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2424d-27ca-4dd8-8f41-fce9c3fa53e6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.462601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2424d-27ca-4dd8-8f41-fce9c3fa53e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.462878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2424d-27ca-4dd8-8f41-fce9c3fa53e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.463623 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2424d-27ca-4dd8-8f41-fce9c3fa53e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-9d0568c9-0721-4382-bf59-93ae1a0bcbdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:19.463817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:19.464648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:19.466182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:19.466182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:19.466182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:19.466182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:19.466427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2424d-27ca-4dd8-8f41-fce9c3fa53e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:19.466507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 368/741] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:19.466507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0568c9-0721-4382-bf59-93ae1a0bcbdb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:19.469637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2424d-27ca-4dd8-8f41-fce9c3fa53e6 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:19.483805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 374/742] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:19.483805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2424d-27ca-4dd8-8f41-fce9c3fa53e6 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:19.572936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0568c9-0721-4382-bf59-93ae1a0bcbdb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:19.582760 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:19.589240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2424d-27ca-4dd8-8f41-fce9c3fa53e6 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:19.609235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62721e6a-306f-49a2-93dc-864f5a477ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:19.609900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62721e6a-306f-49a2-93dc-864f5a477ba6 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.610181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62721e6a-306f-49a2-93dc-864f5a477ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.610452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62721e6a-306f-49a2-93dc-864f5a477ba6 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.611172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62721e6a-306f-49a2-93dc-864f5a477ba6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:19.613210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62721e6a-306f-49a2-93dc-864f5a477ba6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:19.624855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb6f7-be3c-4338-b347-af5bfac7f7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:19.625505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb6f7-be3c-4338-b347-af5bfac7f7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.625786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb6f7-be3c-4338-b347-af5bfac7f7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.626060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb6f7-be3c-4338-b347-af5bfac7f7d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.626781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebb6f7-be3c-4338-b347-af5bfac7f7d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:19.627112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebb6f7-be3c-4338-b347-af5bfac7f7d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:19.628495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebb6f7-be3c-4338-b347-af5bfac7f7d1 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:19.639643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 375/743] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:19.640432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb6f7-be3c-4338-b347-af5bfac7f7d1 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:19.684373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 369/744] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:51:19 2022] POST /identity/v3/services => generated 240 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:19.690330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62721e6a-306f-49a2-93dc-864f5a477ba6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:19.708133 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:19.749386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb6f7-be3c-4338-b347-af5bfac7f7d1 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:19.796136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db8282-4218-48cf-b764-2a86d477535a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:19.796840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db8282-4218-48cf-b764-2a86d477535a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.797129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db8282-4218-48cf-b764-2a86d477535a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.797408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db8282-4218-48cf-b764-2a86d477535a None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.798146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db8282-4218-48cf-b764-2a86d477535a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:19.798514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db8282-4218-48cf-b764-2a86d477535a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:19.800604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db8282-4218-48cf-b764-2a86d477535a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:19.818969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 376/745] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:19.819983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db8282-4218-48cf-b764-2a86d477535a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:19.836669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:19.837111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62721e6a-306f-49a2-93dc-864f5a477ba6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:19.875171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed13a4b-e1e9-4dbc-81cc-5ae5ed424039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:19.875971 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed13a4b-e1e9-4dbc-81cc-5ae5ed424039 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:19.876254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed13a4b-e1e9-4dbc-81cc-5ae5ed424039 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:19.876529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed13a4b-e1e9-4dbc-81cc-5ae5ed424039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:19.887482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed13a4b-e1e9-4dbc-81cc-5ae5ed424039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:19.888983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed13a4b-e1e9-4dbc-81cc-5ae5ed424039 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-1ed13a4b-e1e9-4dbc-81cc-5ae5ed424039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:19.906475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:19.907413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:19.909063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:19.909063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:19.909063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:19.909063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:19.909063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 370/746] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:19.909414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed13a4b-e1e9-4dbc-81cc-5ae5ed424039 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:19.972110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db8282-4218-48cf-b764-2a86d477535a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:19.975644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:20.006256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5547b-6048-4a2d-9259-c19e8416f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:20.007088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5547b-6048-4a2d-9259-c19e8416f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.007476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5547b-6048-4a2d-9259-c19e8416f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.007752 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5547b-6048-4a2d-9259-c19e8416f3f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.015444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5547b-6048-4a2d-9259-c19e8416f3f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:20.016895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5547b-6048-4a2d-9259-c19e8416f3f5 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:20.020788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed13a4b-e1e9-4dbc-81cc-5ae5ed424039 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-18d5547b-6048-4a2d-9259-c19e8416f3f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:20.031198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:20.032425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:20.033967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 377/747] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:20.034563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef6428-9fba-4cf3-b825-34d397c6b481 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.034643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef6428-9fba-4cf3-b825-34d397c6b481 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.034852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef6428-9fba-4cf3-b825-34d397c6b481 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.036237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 378/748] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:20.038566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef6428-9fba-4cf3-b825-34d397c6b481 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:20.065015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118d52e-490b-4523-a45b-65ab04aca222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:20.065861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118d52e-490b-4523-a45b-65ab04aca222 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.066147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118d52e-490b-4523-a45b-65ab04aca222 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.066468 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118d52e-490b-4523-a45b-65ab04aca222 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.073391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118d52e-490b-4523-a45b-65ab04aca222 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:20.074840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118d52e-490b-4523-a45b-65ab04aca222 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-d118d52e-490b-4523-a45b-65ab04aca222 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:20.089790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:20.090643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:20.091547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:20.091547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:20.091547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:20.091547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:20.091547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:20.091547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:20.091547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:20.091547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:20.095711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 371/749] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:20.097030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118d52e-490b-4523-a45b-65ab04aca222 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:20.119533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:20.155373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef6428-9fba-4cf3-b825-34d397c6b481 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:20.190865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd9af2-0f06-4148-b8e7-3c16954f210a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:20.191526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd9af2-0f06-4148-b8e7-3c16954f210a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.191801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd9af2-0f06-4148-b8e7-3c16954f210a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.192084 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd9af2-0f06-4148-b8e7-3c16954f210a None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.192831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd9af2-0f06-4148-b8e7-3c16954f210a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:20.193141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd9af2-0f06-4148-b8e7-3c16954f210a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:20.194501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd9af2-0f06-4148-b8e7-3c16954f210a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:20.206303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118d52e-490b-4523-a45b-65ab04aca222 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:20.212908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 379/750] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:20.214260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5824b5-ce3f-4f75-bc52-ec2f05fc7108 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.214550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5824b5-ce3f-4f75-bc52-ec2f05fc7108 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.214825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5824b5-ce3f-4f75-bc52-ec2f05fc7108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.228117 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:20.233172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-0e5824b5-ce3f-4f75-bc52-ec2f05fc7108 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:20.234375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0e5824b5-ce3f-4f75-bc52-ec2f05fc7108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:20.242296 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1d997-b867-417e-a0ff-58061cf076e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:20.242974 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1d997-b867-417e-a0ff-58061cf076e5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.243265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1d997-b867-417e-a0ff-58061cf076e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.243542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1d997-b867-417e-a0ff-58061cf076e5 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.244395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1d997-b867-417e-a0ff-58061cf076e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:20.244706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1d997-b867-417e-a0ff-58061cf076e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:20.246167 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1d997-b867-417e-a0ff-58061cf076e5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:20.269119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 372/751] 10.222.0.45 () {56 vars in 1135 bytes} [Tue Jul 5 15:51:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:20.272294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1d997-b867-417e-a0ff-58061cf076e5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:20.320788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 380/752] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:20.320788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5824b5-ce3f-4f75-bc52-ec2f05fc7108 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:20.373302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:20.397905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1d997-b867-417e-a0ff-58061cf076e5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:20.415801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5824b5-ce3f-4f75-bc52-ec2f05fc7108 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:20.433472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2f170-1189-4a81-9a99-591c9537416b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:20.434160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2f170-1189-4a81-9a99-591c9537416b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.434493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2f170-1189-4a81-9a99-591c9537416b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.434773 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2f170-1189-4a81-9a99-591c9537416b None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.435564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2f170-1189-4a81-9a99-591c9537416b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:20.435871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2f170-1189-4a81-9a99-591c9537416b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:20.437343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2f170-1189-4a81-9a99-591c9537416b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:20.455631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 373/753] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:20.455822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75fd48-e85f-4fca-bd99-cd6a66d003a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:20.455933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2f170-1189-4a81-9a99-591c9537416b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:20.456017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75fd48-e85f-4fca-bd99-cd6a66d003a4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.456017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75fd48-e85f-4fca-bd99-cd6a66d003a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.456017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75fd48-e85f-4fca-bd99-cd6a66d003a4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.464151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc75fd48-e85f-4fca-bd99-cd6a66d003a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:20.465621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc75fd48-e85f-4fca-bd99-cd6a66d003a4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-bc75fd48-e85f-4fca-bd99-cd6a66d003a4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:20.482639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:20.483606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 381/754] 10.222.0.45 () {56 vars in 1117 bytes} [Tue Jul 5 15:51:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:20.484561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75fd48-e85f-4fca-bd99-cd6a66d003a4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:20.504191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:20.567107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2f170-1189-4a81-9a99-591c9537416b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:20.590400 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75fd48-e85f-4fca-bd99-cd6a66d003a4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:20.603124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c6815-f07e-476d-ad0e-c481f92a98ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:20.603767 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c6815-f07e-476d-ad0e-c481f92a98ae None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.604054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c6815-f07e-476d-ad0e-c481f92a98ae None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.604332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c6815-f07e-476d-ad0e-c481f92a98ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:20.625723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda325e-fc5d-449d-883d-03e5e07ee37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:20.626451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda325e-fc5d-449d-883d-03e5e07ee37c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.626741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda325e-fc5d-449d-883d-03e5e07ee37c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.627548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda325e-fc5d-449d-883d-03e5e07ee37c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.628193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda325e-fc5d-449d-883d-03e5e07ee37c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:20.629572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda325e-fc5d-449d-883d-03e5e07ee37c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-5cda325e-fc5d-449d-883d-03e5e07ee37c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:20.641670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:20.643409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:20.649416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 382/755] 10.222.0.45 () {56 vars in 1129 bytes} [Tue Jul 5 15:51:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:20.650458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda325e-fc5d-449d-883d-03e5e07ee37c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:20.682396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c6815-f07e-476d-ad0e-c481f92a98ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:20.683840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c6815-f07e-476d-ad0e-c481f92a98ae None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:20.725284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:20.750996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda325e-fc5d-449d-883d-03e5e07ee37c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:20.787158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9479-7ff7-481c-b550-8ceb592ccb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:20.788041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9479-7ff7-481c-b550-8ceb592ccb30 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.788325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9479-7ff7-481c-b550-8ceb592ccb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.788604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9479-7ff7-481c-b550-8ceb592ccb30 None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/ef95ab7b924d456195a8761e894d911b/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.790630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9479-7ff7-481c-b550-8ceb592ccb30 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=ef95ab7b924d456195a8761e894d911b, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:20.792067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9479-7ff7-481c-b550-8ceb592ccb30 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:20.793468 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 374/756] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:51:20.794086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c6815-f07e-476d-ad0e-c481f92a98ae None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:20.856819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 383/757] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:51:20 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/ef95ab7b924d456195a8761e894d911b/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:51:20.858083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9479-7ff7-481c-b550-8ceb592ccb30 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:20.922966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c6815-f07e-476d-ad0e-c481f92a98ae None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:20.936646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:20.965385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e089404-9420-4c57-82c1-2fbe47f2dd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:20.966169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e089404-9420-4c57-82c1-2fbe47f2dd94 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:20.966527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e089404-9420-4c57-82c1-2fbe47f2dd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:20.966912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e089404-9420-4c57-82c1-2fbe47f2dd94 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:20.967905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e089404-9420-4c57-82c1-2fbe47f2dd94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:20.968311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e089404-9420-4c57-82c1-2fbe47f2dd94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:20.969949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e089404-9420-4c57-82c1-2fbe47f2dd94 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:20.980889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:20.992602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 375/758] 10.222.0.45 () {56 vars in 1149 bytes} [Tue Jul 5 15:51:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:20.993775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e089404-9420-4c57-82c1-2fbe47f2dd94 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:21.001206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9479-7ff7-481c-b550-8ceb592ccb30 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:21.042708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc06a0-7f4b-4ab2-88e0-fb22d75b4f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:21.043376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc06a0-7f4b-4ab2-88e0-fb22d75b4f3b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:21.043662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc06a0-7f4b-4ab2-88e0-fb22d75b4f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:21.043945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc06a0-7f4b-4ab2-88e0-fb22d75b4f3b None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:21.044655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc06a0-7f4b-4ab2-88e0-fb22d75b4f3b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:21.044995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc06a0-7f4b-4ab2-88e0-fb22d75b4f3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:21.046406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc06a0-7f4b-4ab2-88e0-fb22d75b4f3b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:21.052501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 384/759] 10.222.0.45 () {56 vars in 1124 bytes} [Tue Jul 5 15:51:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:21.053318 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc06a0-7f4b-4ab2-88e0-fb22d75b4f3b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:21.118753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e089404-9420-4c57-82c1-2fbe47f2dd94 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:21.162114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41f172-1822-43d8-a2cf-c5a7df0f8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:21.162911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41f172-1822-43d8-a2cf-c5a7df0f8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:21.163189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41f172-1822-43d8-a2cf-c5a7df0f8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:21.163459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41f172-1822-43d8-a2cf-c5a7df0f8ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:21.179148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc06a0-7f4b-4ab2-88e0-fb22d75b4f3b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:21.179310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41f172-1822-43d8-a2cf-c5a7df0f8ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:21.180927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41f172-1822-43d8-a2cf-c5a7df0f8ee4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:21.189476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7d41f172-1822-43d8-a2cf-c5a7df0f8ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:21.203281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:21.204388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:21.205550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:21.205550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:21.205550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:21.205550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:21.207388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 376/760] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:21.212954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41f172-1822-43d8-a2cf-c5a7df0f8ee4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:21.219787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad333a-3e36-4aaf-911f-eece8fe9ca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:21.221211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad333a-3e36-4aaf-911f-eece8fe9ca48 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:21.221836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad333a-3e36-4aaf-911f-eece8fe9ca48 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:21.222447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad333a-3e36-4aaf-911f-eece8fe9ca48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:21.285122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:21.322083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad333a-3e36-4aaf-911f-eece8fe9ca48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:21.323802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad333a-3e36-4aaf-911f-eece8fe9ca48 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:21.330673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41f172-1822-43d8-a2cf-c5a7df0f8ee4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:21.367634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e118c-f388-4da6-9b5b-cf3469594c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:21.368654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e118c-f388-4da6-9b5b-cf3469594c95 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:21.369183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e118c-f388-4da6-9b5b-cf3469594c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:21.369641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e118c-f388-4da6-9b5b-cf3469594c95 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:21.370535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e118c-f388-4da6-9b5b-cf3469594c95 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:21.371020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e118c-f388-4da6-9b5b-cf3469594c95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:21.372572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e118c-f388-4da6-9b5b-cf3469594c95 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:21.378660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 377/761] 10.222.0.45 () {56 vars in 1124 bytes} [Tue Jul 5 15:51:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:21.379775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e118c-f388-4da6-9b5b-cf3469594c95 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:21.465572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 385/762] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:51:21.466980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad333a-3e36-4aaf-911f-eece8fe9ca48 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:21.481516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:21.494226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e118c-f388-4da6-9b5b-cf3469594c95 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:21.508141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:21.532609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f4527-b4e8-4f70-969d-4a1b29c5aa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:21.534210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f4527-b4e8-4f70-969d-4a1b29c5aa24 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:21.534642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f4527-b4e8-4f70-969d-4a1b29c5aa24 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:21.535067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f4527-b4e8-4f70-969d-4a1b29c5aa24 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:21.537430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f4527-b4e8-4f70-969d-4a1b29c5aa24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:21.537430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f4527-b4e8-4f70-969d-4a1b29c5aa24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:21.538066 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f4527-b4e8-4f70-969d-4a1b29c5aa24 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:21.556660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 378/763] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:21.557854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f4527-b4e8-4f70-969d-4a1b29c5aa24 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:21.598789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad333a-3e36-4aaf-911f-eece8fe9ca48 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:21.635839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7310b-c6b8-4014-b15d-52248d443d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:21.636748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7310b-c6b8-4014-b15d-52248d443d94 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:21.637197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7310b-c6b8-4014-b15d-52248d443d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:21.637613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7310b-c6b8-4014-b15d-52248d443d94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:21.638566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7310b-c6b8-4014-b15d-52248d443d94 None admin] RBAC: Adding query filter params (interface=public, service_id=edcb1ccfbe734175a1474d81b298d765, region_id=RegionOne) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:21.639025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7310b-c6b8-4014-b15d-52248d443d94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:21.640530 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7310b-c6b8-4014-b15d-52248d443d94 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:21.649612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 386/764] 10.222.0.45 () {56 vars in 1251 bytes} [Tue Jul 5 15:51:21 2022] GET /identity/v3/endpoints?service_id=edcb1ccfbe734175a1474d81b298d765&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:21.650622 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7310b-c6b8-4014-b15d-52248d443d94 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:21.685371 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f4527-b4e8-4f70-969d-4a1b29c5aa24 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:21.723100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd055aa-d3fd-4a56-8fa7-bc6902773616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:21.724070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd055aa-d3fd-4a56-8fa7-bc6902773616 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:21.724349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd055aa-d3fd-4a56-8fa7-bc6902773616 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:21.725046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd055aa-d3fd-4a56-8fa7-bc6902773616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:21.737445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:21.741613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd055aa-d3fd-4a56-8fa7-bc6902773616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:21.744052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd055aa-d3fd-4a56-8fa7-bc6902773616 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-9dd055aa-d3fd-4a56-8fa7-bc6902773616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:21.765784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:21.766735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:21.768199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:21.768199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:21.768199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:21.768199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:21.768429 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7310b-c6b8-4014-b15d-52248d443d94 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:21.771957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 379/765] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:21.772825 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd055aa-d3fd-4a56-8fa7-bc6902773616 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:21.799201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:21.803354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8e4e2-b320-43dd-971d-cb916883e954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:21.804141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8e4e2-b320-43dd-971d-cb916883e954 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:21.804493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8e4e2-b320-43dd-971d-cb916883e954 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:21.804875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8e4e2-b320-43dd-971d-cb916883e954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:21.811787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8e4e2-b320-43dd-971d-cb916883e954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:21.813354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8e4e2-b320-43dd-971d-cb916883e954 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-2cf8e4e2-b320-43dd-971d-cb916883e954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:21.827516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:21.828566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:21.829560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:21.829560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:21.829560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:21.829560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:21.829560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:21.829560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:21.829560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:21.829560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:21.829560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 387/766] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:21.830855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d01d1d-9d93-4ce2-ba43-73e0e6f935bd None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:21.831139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d01d1d-9d93-4ce2-ba43-73e0e6f935bd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:21.831414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d01d1d-9d93-4ce2-ba43-73e0e6f935bd None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:21.836808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 388/767] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:21.836808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d01d1d-9d93-4ce2-ba43-73e0e6f935bd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:21.885760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd055aa-d3fd-4a56-8fa7-bc6902773616 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:21.922549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6b2e8-ae85-4ca8-b0ca-c1131dd0b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:21.923441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6b2e8-ae85-4ca8-b0ca-c1131dd0b1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:21.923753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6b2e8-ae85-4ca8-b0ca-c1131dd0b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:21.924030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6b2e8-ae85-4ca8-b0ca-c1131dd0b1f2 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:21.924716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6b2e8-ae85-4ca8-b0ca-c1131dd0b1f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:21.926120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6b2e8-ae85-4ca8-b0ca-c1131dd0b1f2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:21.936855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 380/768] 10.222.0.45 () {56 vars in 1087 bytes} [Tue Jul 5 15:51:21 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:21.936855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6b2e8-ae85-4ca8-b0ca-c1131dd0b1f2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:21.959849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d01d1d-9d93-4ce2-ba43-73e0e6f935bd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:21.993313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:21.998838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ecb24-9eac-4b56-b824-80e3aa23fca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:21.999520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecb24-9eac-4b56-b824-80e3aa23fca2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.003601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecb24-9eac-4b56-b824-80e3aa23fca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.003601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ecb24-9eac-4b56-b824-80e3aa23fca2 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.003601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ecb24-9eac-4b56-b824-80e3aa23fca2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:22.003601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ecb24-9eac-4b56-b824-80e3aa23fca2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:22.003601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ecb24-9eac-4b56-b824-80e3aa23fca2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:22.021191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 389/769] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:22.022900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ceb4b-0bd3-4831-a113-46c30e7b7c42 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.023190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ceb4b-0bd3-4831-a113-46c30e7b7c42 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.023473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ceb4b-0bd3-4831-a113-46c30e7b7c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.042707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-2c5ceb4b-0bd3-4831-a113-46c30e7b7c42 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:22.044048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2c5ceb4b-0bd3-4831-a113-46c30e7b7c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:22.052947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:22.074341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6b2e8-ae85-4ca8-b0ca-c1131dd0b1f2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:22.118056 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1e639-89d0-41b5-8c7b-6a11ce9b0ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:22.119644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1e639-89d0-41b5-8c7b-6a11ce9b0ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.119934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1e639-89d0-41b5-8c7b-6a11ce9b0ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.120364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1e639-89d0-41b5-8c7b-6a11ce9b0ec0 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.121490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1e639-89d0-41b5-8c7b-6a11ce9b0ec0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:22.121807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1e639-89d0-41b5-8c7b-6a11ce9b0ec0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:22.124566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1e639-89d0-41b5-8c7b-6a11ce9b0ec0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:22.131505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 390/770] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:22.131505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ceb4b-0bd3-4831-a113-46c30e7b7c42 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:22.139362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 381/771] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:22.145479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1e639-89d0-41b5-8c7b-6a11ce9b0ec0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:22.252751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ceb4b-0bd3-4831-a113-46c30e7b7c42 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:22.260858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1e639-89d0-41b5-8c7b-6a11ce9b0ec0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:22.282781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:22.296191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a76bd-2e1d-4d8c-8347-cf22b538f104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:22.297174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a76bd-2e1d-4d8c-8347-cf22b538f104 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.297354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a76bd-2e1d-4d8c-8347-cf22b538f104 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.297618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a76bd-2e1d-4d8c-8347-cf22b538f104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.298565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a76bd-2e1d-4d8c-8347-cf22b538f104 None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=8da9cdd70af646928ee0edcadbcb524f) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:22.299521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a76bd-2e1d-4d8c-8347-cf22b538f104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:22.300485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57283-45c8-4955-bc02-46fb02ea7835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:22.301014 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a76bd-2e1d-4d8c-8347-cf22b538f104 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:22.301440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57283-45c8-4955-bc02-46fb02ea7835 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.301788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57283-45c8-4955-bc02-46fb02ea7835 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.302122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57283-45c8-4955-bc02-46fb02ea7835 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.307336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 382/772] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:22 2022] GET /identity/v3/role_assignments?user.id=8da9cdd70af646928ee0edcadbcb524f&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:22.308104 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a76bd-2e1d-4d8c-8347-cf22b538f104 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:22.317372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c57283-45c8-4955-bc02-46fb02ea7835 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:22.318878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c57283-45c8-4955-bc02-46fb02ea7835 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-50c57283-45c8-4955-bc02-46fb02ea7835 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:22.337550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:22.338674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:22.339781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:22.339781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:22.339781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:22.339781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:22.339781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:22.339781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:22.339781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:22.339781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:22.339781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 391/773] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:22.349158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:22.350075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57283-45c8-4955-bc02-46fb02ea7835 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:22.485984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a76bd-2e1d-4d8c-8347-cf22b538f104 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:22.519698 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57283-45c8-4955-bc02-46fb02ea7835 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:22.526073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da0a8d-9f41-4c3e-8d12-7e16a8eb5420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:22.527886 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da0a8d-9f41-4c3e-8d12-7e16a8eb5420 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.528518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da0a8d-9f41-4c3e-8d12-7e16a8eb5420 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.529025 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da0a8d-9f41-4c3e-8d12-7e16a8eb5420 None admin] PATH_INFO: `/v3/services/network` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.529893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da0a8d-9f41-4c3e-8d12-7e16a8eb5420 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:22.531671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da0a8d-9f41-4c3e-8d12-7e16a8eb5420 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-69da0a8d-9f41-4c3e-8d12-7e16a8eb5420 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:22.545062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:22.546078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:51:22.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:22.550336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 383/774] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:22 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:22.556360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e484b-43f8-4212-81fa-d851c33a39ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:22.556655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da0a8d-9f41-4c3e-8d12-7e16a8eb5420 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:22.557061 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e484b-43f8-4212-81fa-d851c33a39ec None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.557343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e484b-43f8-4212-81fa-d851c33a39ec None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.557615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e484b-43f8-4212-81fa-d851c33a39ec None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.558516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e484b-43f8-4212-81fa-d851c33a39ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:22.558827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e484b-43f8-4212-81fa-d851c33a39ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:22.560257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e484b-43f8-4212-81fa-d851c33a39ec None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:22.574971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 392/775] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:22.583742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e484b-43f8-4212-81fa-d851c33a39ec None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:22.632114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:22.645688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:22.669803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da0a8d-9f41-4c3e-8d12-7e16a8eb5420 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:22.692289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e484b-43f8-4212-81fa-d851c33a39ec None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:22.714707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21994999-25f5-4ffd-9df0-101a57f370e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:22.715635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21994999-25f5-4ffd-9df0-101a57f370e9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.715998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21994999-25f5-4ffd-9df0-101a57f370e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.716332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21994999-25f5-4ffd-9df0-101a57f370e9 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.717180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21994999-25f5-4ffd-9df0-101a57f370e9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:22.717685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21994999-25f5-4ffd-9df0-101a57f370e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:22.719279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21994999-25f5-4ffd-9df0-101a57f370e9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:22.725508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 384/776] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:22.730886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb83fe-f4e8-4fa7-9c54-6f2099e0d2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:22.731364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21994999-25f5-4ffd-9df0-101a57f370e9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:22.732004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb83fe-f4e8-4fa7-9c54-6f2099e0d2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.732456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb83fe-f4e8-4fa7-9c54-6f2099e0d2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.732992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb83fe-f4e8-4fa7-9c54-6f2099e0d2e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.733967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb83fe-f4e8-4fa7-9c54-6f2099e0d2e8 None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=7245effe463b45dfa946ca2eda2e0c98) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:22.734521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb83fe-f4e8-4fa7-9c54-6f2099e0d2e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:22.736757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb83fe-f4e8-4fa7-9c54-6f2099e0d2e8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:22.743293 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 393/777] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:22 2022] GET /identity/v3/role_assignments?user.id=7245effe463b45dfa946ca2eda2e0c98&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:22.847667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21994999-25f5-4ffd-9df0-101a57f370e9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:22.887792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1441d4-e0fc-403a-a367-daed4975a5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:22.888626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1441d4-e0fc-403a-a367-daed4975a5dd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.889092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1441d4-e0fc-403a-a367-daed4975a5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.889472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1441d4-e0fc-403a-a367-daed4975a5dd None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.890301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1441d4-e0fc-403a-a367-daed4975a5dd None admin] RBAC: Adding query filter params (type=network) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:22.890730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1441d4-e0fc-403a-a367-daed4975a5dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:22.892238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1441d4-e0fc-403a-a367-daed4975a5dd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:22.898708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 385/778] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:22.904461 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb83fe-f4e8-4fa7-9c54-6f2099e0d2e8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:22.981038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cbc95-0834-4a1c-a415-ba97e1dac2ea None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.981364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cbc95-0834-4a1c-a415-ba97e1dac2ea None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.981606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cbc95-0834-4a1c-a415-ba97e1dac2ea None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:22.983093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 386/779] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:22.988451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01430e2-1e57-4183-a4c3-e8e30c3e3c81 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:22.988721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01430e2-1e57-4183-a4c3-e8e30c3e3c81 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:22.989031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01430e2-1e57-4183-a4c3-e8e30c3e3c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:23.003231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:23.014994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb83fe-f4e8-4fa7-9c54-6f2099e0d2e8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:23.022303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c01430e2-1e57-4183-a4c3-e8e30c3e3c81 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:23.023518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c01430e2-1e57-4183-a4c3-e8e30c3e3c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:23.025107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:23.079148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36427803-ec72-4fa5-86d1-229ac5742047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:23.080504 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36427803-ec72-4fa5-86d1-229ac5742047 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:23.080616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36427803-ec72-4fa5-86d1-229ac5742047 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:23.080919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36427803-ec72-4fa5-86d1-229ac5742047 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:23.081715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36427803-ec72-4fa5-86d1-229ac5742047 None admin] RBAC: Adding query filter params (interface=public, service_id=69a5460bb5e94383964e921f0e96d61a, region_id=RegionOne) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:23.082026 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36427803-ec72-4fa5-86d1-229ac5742047 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:23.083418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36427803-ec72-4fa5-86d1-229ac5742047 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:23.096991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 394/780] 10.222.0.45 () {56 vars in 1251 bytes} [Tue Jul 5 15:51:22 2022] GET /identity/v3/endpoints?service_id=69a5460bb5e94383964e921f0e96d61a&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:23.102619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36427803-ec72-4fa5-86d1-229ac5742047 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:23.127787 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 387/781] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:23.163295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01430e2-1e57-4183-a4c3-e8e30c3e3c81 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:23.274590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01430e2-1e57-4183-a4c3-e8e30c3e3c81 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:23.312256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399f897-50f8-4de5-a4ea-186067bc86bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:23.313175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399f897-50f8-4de5-a4ea-186067bc86bb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:23.313950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399f897-50f8-4de5-a4ea-186067bc86bb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:23.314345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399f897-50f8-4de5-a4ea-186067bc86bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:23.316416 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:23.326067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399f897-50f8-4de5-a4ea-186067bc86bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-2399f897-50f8-4de5-a4ea-186067bc86bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:23.335419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:23.336360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:23.337249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 388/782] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:23.346312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399f897-50f8-4de5-a4ea-186067bc86bb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:23.377456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36427803-ec72-4fa5-86d1-229ac5742047 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:23.447048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15844f17-3842-4031-a55d-d4aaeec0e445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:23.448159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15844f17-3842-4031-a55d-d4aaeec0e445 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:23.450177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15844f17-3842-4031-a55d-d4aaeec0e445 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:23.451330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15844f17-3842-4031-a55d-d4aaeec0e445 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:23.452227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15844f17-3842-4031-a55d-d4aaeec0e445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:23.453969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15844f17-3842-4031-a55d-d4aaeec0e445 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:23.463676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 395/783] 10.222.0.45 () {56 vars in 1087 bytes} [Tue Jul 5 15:51:23 2022] GET /identity/v3/services => generated 772 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:23.500028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399f897-50f8-4de5-a4ea-186067bc86bb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:23.552166 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5f37b-263a-43be-b27e-0819d0bb7288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:23.552891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5f37b-263a-43be-b27e-0819d0bb7288 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:23.553180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5f37b-263a-43be-b27e-0819d0bb7288 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:23.553458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5f37b-263a-43be-b27e-0819d0bb7288 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:23.554233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5f37b-263a-43be-b27e-0819d0bb7288 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:23.554543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5f37b-263a-43be-b27e-0819d0bb7288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:23.555992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5f37b-263a-43be-b27e-0819d0bb7288 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:23.569465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 389/784] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:23.574675 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15844f17-3842-4031-a55d-d4aaeec0e445 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:23.625648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:23.712143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15844f17-3842-4031-a55d-d4aaeec0e445 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:23.750810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cd1ab-7397-4cd0-b574-addbb5111dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:23.751992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cd1ab-7397-4cd0-b574-addbb5111dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:23.752473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cd1ab-7397-4cd0-b574-addbb5111dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:23.753024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cd1ab-7397-4cd0-b574-addbb5111dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:23.766190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5cd1ab-7397-4cd0-b574-addbb5111dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:23.768418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5cd1ab-7397-4cd0-b574-addbb5111dd0 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-af5cd1ab-7397-4cd0-b574-addbb5111dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:23.788473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:23.789601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:23.790473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:23.790473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:23.790473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:23.790473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:23.792597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 396/785] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:23.798141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5f37b-263a-43be-b27e-0819d0bb7288 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:23.869475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:23.927419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5f37b-263a-43be-b27e-0819d0bb7288 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:23.980366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e3cb1-3bee-48ed-aca4-70065c0f9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:23.980984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e3cb1-3bee-48ed-aca4-70065c0f9240 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:23.981271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e3cb1-3bee-48ed-aca4-70065c0f9240 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:23.981550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e3cb1-3bee-48ed-aca4-70065c0f9240 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:23.982280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e3cb1-3bee-48ed-aca4-70065c0f9240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:23.982589 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e3cb1-3bee-48ed-aca4-70065c0f9240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:23.984023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e3cb1-3bee-48ed-aca4-70065c0f9240 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:23.997567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 390/786] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:24.003031 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cd1ab-7397-4cd0-b574-addbb5111dd0 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:24.113579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:24.128812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cd1ab-7397-4cd0-b574-addbb5111dd0 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:24.163604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6b458-f98e-450e-999b-5252fa62816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:24.164397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6b458-f98e-450e-999b-5252fa62816e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:24.169133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6b458-f98e-450e-999b-5252fa62816e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:24.169414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6b458-f98e-450e-999b-5252fa62816e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:24.176877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6b458-f98e-450e-999b-5252fa62816e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:24.178315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6b458-f98e-450e-999b-5252fa62816e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-d0a6b458-f98e-450e-999b-5252fa62816e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:24.199404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:24.200392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:24.201376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:24.201376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:24.201376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:24.201376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:24.201376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:24.201376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:24.201376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:24.201376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:24.201376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:24.201376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:24.203951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 397/787] 10.222.0.45 () {56 vars in 1093 bytes} [Tue Jul 5 15:51:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:24.210332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e3cb1-3bee-48ed-aca4-70065c0f9240 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:24.345847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e3cb1-3bee-48ed-aca4-70065c0f9240 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:24.372079 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:24.400224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2004b40-a4ff-4357-9259-1632c54673bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:24.401276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2004b40-a4ff-4357-9259-1632c54673bd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:24.401770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2004b40-a4ff-4357-9259-1632c54673bd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:24.402241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2004b40-a4ff-4357-9259-1632c54673bd None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:24.403310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2004b40-a4ff-4357-9259-1632c54673bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:24.403814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2004b40-a4ff-4357-9259-1632c54673bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:24.405490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2004b40-a4ff-4357-9259-1632c54673bd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:24.430962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 391/788] 10.222.0.45 () {56 vars in 1133 bytes} [Tue Jul 5 15:51:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:24.436714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6b458-f98e-450e-999b-5252fa62816e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:24.603906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6b458-f98e-450e-999b-5252fa62816e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:24.641055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925d95f-c9a3-40d8-90b0-b1a29841087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:24.641714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925d95f-c9a3-40d8-90b0-b1a29841087f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:24.642005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925d95f-c9a3-40d8-90b0-b1a29841087f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:24.642315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925d95f-c9a3-40d8-90b0-b1a29841087f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:24.657968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:24.672953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde587a0-87f1-4490-ae6d-8ef40b58a157 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:24.673254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde587a0-87f1-4490-ae6d-8ef40b58a157 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:24.674033 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde587a0-87f1-4490-ae6d-8ef40b58a157 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:24.675643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 392/789] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:24.680527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1a276-994e-4064-a917-d72e0968b79c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:24.680969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1a276-994e-4064-a917-d72e0968b79c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:24.681254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1a276-994e-4064-a917-d72e0968b79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:24.709854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-aad1a276-994e-4064-a917-d72e0968b79c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:24.711063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-aad1a276-994e-4064-a917-d72e0968b79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:24.760238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925d95f-c9a3-40d8-90b0-b1a29841087f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:24.761720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b925d95f-c9a3-40d8-90b0-b1a29841087f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:24.802185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 393/790] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:24.832577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1a276-994e-4064-a917-d72e0968b79c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:24.899173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 398/791] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:51:24.904825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925d95f-c9a3-40d8-90b0-b1a29841087f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:24.961306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:25.004645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1a276-994e-4064-a917-d72e0968b79c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:25.007936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:25.023580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b925d95f-c9a3-40d8-90b0-b1a29841087f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:25.047654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993b062-ad23-42a9-977c-22e284681abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:25.048493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993b062-ad23-42a9-977c-22e284681abe None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.049391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993b062-ad23-42a9-977c-22e284681abe None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.049719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993b062-ad23-42a9-977c-22e284681abe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.050385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993b062-ad23-42a9-977c-22e284681abe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:25.051774 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993b062-ad23-42a9-977c-22e284681abe None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:25.060338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb186ac5-5ac5-4e75-9bba-68bdfd484a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:25.061173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb186ac5-5ac5-4e75-9bba-68bdfd484a76 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.061455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb186ac5-5ac5-4e75-9bba-68bdfd484a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.061729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb186ac5-5ac5-4e75-9bba-68bdfd484a76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7993b062-ad23-42a9-977c-22e284681abe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:25.064977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:25.065888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:25.066732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 394/792] 10.222.0.45 () {56 vars in 1129 bytes} [Tue Jul 5 15:51:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:25.068145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36fc26-aecf-4f17-9317-577e43755609 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.068145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36fc26-aecf-4f17-9317-577e43755609 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.068438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36fc26-aecf-4f17-9317-577e43755609 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.072657 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 395/793] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:25.074341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36fc26-aecf-4f17-9317-577e43755609 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:25.081910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb186ac5-5ac5-4e75-9bba-68bdfd484a76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:25.083354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb186ac5-5ac5-4e75-9bba-68bdfd484a76 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-fb186ac5-5ac5-4e75-9bba-68bdfd484a76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:25.102000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:25.104027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:25.106163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:25.106163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:25.106163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:25.106163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:25.106163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 399/794] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:25.106163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c42023-b9a5-4e1d-b331-5c2f51ebab66 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.106163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c42023-b9a5-4e1d-b331-5c2f51ebab66 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.106163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c42023-b9a5-4e1d-b331-5c2f51ebab66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.122536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-82c42023-b9a5-4e1d-b331-5c2f51ebab66 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:25.123802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-82c42023-b9a5-4e1d-b331-5c2f51ebab66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:25.188973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36fc26-aecf-4f17-9317-577e43755609 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:25.202054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 400/795] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:25.202986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c42023-b9a5-4e1d-b331-5c2f51ebab66 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:25.226464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627b9b8-e596-4695-af67-a5d62d5a852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:25.227905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627b9b8-e596-4695-af67-a5d62d5a852f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.228381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627b9b8-e596-4695-af67-a5d62d5a852f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.228885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627b9b8-e596-4695-af67-a5d62d5a852f None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.229626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627b9b8-e596-4695-af67-a5d62d5a852f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:25.229942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627b9b8-e596-4695-af67-a5d62d5a852f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:25.231391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627b9b8-e596-4695-af67-a5d62d5a852f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:25.236725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 396/796] 10.222.0.45 () {56 vars in 1124 bytes} [Tue Jul 5 15:51:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:25.242338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627b9b8-e596-4695-af67-a5d62d5a852f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:25.249980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:25.277244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:25.334049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c42023-b9a5-4e1d-b331-5c2f51ebab66 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:25.352042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627b9b8-e596-4695-af67-a5d62d5a852f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:25.371345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef9f14-2eab-4406-a295-ce6f0346850a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:25.372123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef9f14-2eab-4406-a295-ce6f0346850a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.372406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef9f14-2eab-4406-a295-ce6f0346850a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.372682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef9f14-2eab-4406-a295-ce6f0346850a None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.373472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef9f14-2eab-4406-a295-ce6f0346850a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:25.373781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef9f14-2eab-4406-a295-ce6f0346850a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:25.375400 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef9f14-2eab-4406-a295-ce6f0346850a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:25.385879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939627dd-bbde-4abb-9d30-2d3353404304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:25.386542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939627dd-bbde-4abb-9d30-2d3353404304 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.386821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939627dd-bbde-4abb-9d30-2d3353404304 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.387093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939627dd-bbde-4abb-9d30-2d3353404304 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.387793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939627dd-bbde-4abb-9d30-2d3353404304 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:25.388095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939627dd-bbde-4abb-9d30-2d3353404304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:25.389479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939627dd-bbde-4abb-9d30-2d3353404304 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:25.392723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 401/797] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:25.393709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef9f14-2eab-4406-a295-ce6f0346850a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:25.400383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 397/798] 10.222.0.45 () {56 vars in 1124 bytes} [Tue Jul 5 15:51:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:25.401711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609ecb2-5017-4a65-9f1a-647e07bf1ef1 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.401998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609ecb2-5017-4a65-9f1a-647e07bf1ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.402272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609ecb2-5017-4a65-9f1a-647e07bf1ef1 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.407569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 398/799] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:25.407569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609ecb2-5017-4a65-9f1a-647e07bf1ef1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:25.489647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:25.502765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef9f14-2eab-4406-a295-ce6f0346850a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:25.511116 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:25.515352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609ecb2-5017-4a65-9f1a-647e07bf1ef1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:25.538561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708b122-e858-444c-bce4-8f379dbaac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:25.539840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708b122-e858-444c-bce4-8f379dbaac49 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.539840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708b122-e858-444c-bce4-8f379dbaac49 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.539997 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708b122-e858-444c-bce4-8f379dbaac49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.550523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708b122-e858-444c-bce4-8f379dbaac49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:25.551960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eb040-7d70-4230-8957-340bf124453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:25.552748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb040-7d70-4230-8957-340bf124453c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.553059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb040-7d70-4230-8957-340bf124453c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.553337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eb040-7d70-4230-8957-340bf124453c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-6708b122-e858-444c-bce4-8f379dbaac49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:25.558177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:25.559109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:25.559920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 402/800] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:25.560587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708b122-e858-444c-bce4-8f379dbaac49 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:25.566103 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8eb040-7d70-4230-8957-340bf124453c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:25.568218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8eb040-7d70-4230-8957-340bf124453c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4b8eb040-7d70-4230-8957-340bf124453c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:25.583847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:25.585145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:25.586024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:25.586024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:25.586024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:25.586024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:25.586024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:25.586024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:25.586024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:25.586024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:25.586024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 399/801] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:25.586024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4301e7-252d-43cf-85a7-ef00ec707724 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.586545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4301e7-252d-43cf-85a7-ef00ec707724 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.586545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4301e7-252d-43cf-85a7-ef00ec707724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.604573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5e4301e7-252d-43cf-85a7-ef00ec707724 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:25.605801 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5e4301e7-252d-43cf-85a7-ef00ec707724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:25.670253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708b122-e858-444c-bce4-8f379dbaac49 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:25.680470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 400/802] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:25.681637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4301e7-252d-43cf-85a7-ef00ec707724 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:25.718436 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d5ab5-ed50-4c7c-8386-116fa086bee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:25.719100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d5ab5-ed50-4c7c-8386-116fa086bee7 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.719381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d5ab5-ed50-4c7c-8386-116fa086bee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.719653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d5ab5-ed50-4c7c-8386-116fa086bee7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.720419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d5ab5-ed50-4c7c-8386-116fa086bee7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:25.721836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d5ab5-ed50-4c7c-8386-116fa086bee7 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:25.768752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:25.782965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4301e7-252d-43cf-85a7-ef00ec707724 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:25.815710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636002b8-eece-4695-a97b-101be852c1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:25.816378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636002b8-eece-4695-a97b-101be852c1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:25.816659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636002b8-eece-4695-a97b-101be852c1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:25.816960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636002b8-eece-4695-a97b-101be852c1c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:25.817699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636002b8-eece-4695-a97b-101be852c1c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:25.818000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636002b8-eece-4695-a97b-101be852c1c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:25.819415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636002b8-eece-4695-a97b-101be852c1c8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:25.827454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 403/803] 10.222.0.45 () {60 vars in 1144 bytes} [Tue Jul 5 15:51:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:25.828483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d5ab5-ed50-4c7c-8386-116fa086bee7 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:25.834107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 401/804] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:25.834107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636002b8-eece-4695-a97b-101be852c1c8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:25.841499 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:25.977350 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636002b8-eece-4695-a97b-101be852c1c8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:25.979145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d5ab5-ed50-4c7c-8386-116fa086bee7 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:26.016395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038bd6f-b372-4293-9385-54c220b6141c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.017264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038bd6f-b372-4293-9385-54c220b6141c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.017555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038bd6f-b372-4293-9385-54c220b6141c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.017836 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038bd6f-b372-4293-9385-54c220b6141c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.022169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b63ec3-1378-4f14-bfa1-0e63484d0c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.022169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b63ec3-1378-4f14-bfa1-0e63484d0c2c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.022169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b63ec3-1378-4f14-bfa1-0e63484d0c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.022169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b63ec3-1378-4f14-bfa1-0e63484d0c2c None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.022901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b63ec3-1378-4f14-bfa1-0e63484d0c2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:26.023215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b63ec3-1378-4f14-bfa1-0e63484d0c2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:26.025599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b63ec3-1378-4f14-bfa1-0e63484d0c2c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:26.033973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038bd6f-b372-4293-9385-54c220b6141c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:26.037449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 404/805] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:26.038353 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b63ec3-1378-4f14-bfa1-0e63484d0c2c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8038bd6f-b372-4293-9385-54c220b6141c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:26.042199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:26.043148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:26.043973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 402/806] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:26.045207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e282e-e899-4239-bd4e-ab7f38af62a6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.045207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e282e-e899-4239-bd4e-ab7f38af62a6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.045482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e282e-e899-4239-bd4e-ab7f38af62a6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.046868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 403/807] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:26.047430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e282e-e899-4239-bd4e-ab7f38af62a6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:26.059780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:26.099846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:26.144916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b63ec3-1378-4f14-bfa1-0e63484d0c2c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:26.164012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e282e-e899-4239-bd4e-ab7f38af62a6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:26.181022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e2479-ef2c-4e4d-a5b3-4a94f044b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.181816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e2479-ef2c-4e4d-a5b3-4a94f044b4df None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.182098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e2479-ef2c-4e4d-a5b3-4a94f044b4df None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.182373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e2479-ef2c-4e4d-a5b3-4a94f044b4df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.194389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e2479-ef2c-4e4d-a5b3-4a94f044b4df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:26.195844 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e2479-ef2c-4e4d-a5b3-4a94f044b4df None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:26.201049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c43ca7-17ad-492c-b294-b8385e9922d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.201913 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43ca7-17ad-492c-b294-b8385e9922d1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.202747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43ca7-17ad-492c-b294-b8385e9922d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.203209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43ca7-17ad-492c-b294-b8385e9922d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.204643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c43ca7-17ad-492c-b294-b8385e9922d1 None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=ef95ab7b924d456195a8761e894d911b) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:26.205191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c43ca7-17ad-492c-b294-b8385e9922d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:26.206865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c43ca7-17ad-492c-b294-b8385e9922d1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-235e2479-ef2c-4e4d-a5b3-4a94f044b4df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:26.212840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:26.213731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:26.214708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:26.214708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:26.214708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:26.214708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:26.225033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 405/808] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:26.225033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e2479-ef2c-4e4d-a5b3-4a94f044b4df None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:26.225292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 404/809] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/role_assignments?user.id=ef95ab7b924d456195a8761e894d911b&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:26.225292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c247-b469-40e4-a6ce-029106ee7674 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.225292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c247-b469-40e4-a6ce-029106ee7674 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.225292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c247-b469-40e4-a6ce-029106ee7674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.244833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9841c247-b469-40e4-a6ce-029106ee7674 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:26.246078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9841c247-b469-40e4-a6ce-029106ee7674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:26.335092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 405/810] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:26.336391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c247-b469-40e4-a6ce-029106ee7674 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:26.343540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:26.363841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e2479-ef2c-4e4d-a5b3-4a94f044b4df None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:26.381112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:26.406981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039a3a3-9697-45b9-b379-3c9fc0f0b8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.408702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039a3a3-9697-45b9-b379-3c9fc0f0b8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.409086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039a3a3-9697-45b9-b379-3c9fc0f0b8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.409452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039a3a3-9697-45b9-b379-3c9fc0f0b8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.411019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039a3a3-9697-45b9-b379-3c9fc0f0b8c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:26.411133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039a3a3-9697-45b9-b379-3c9fc0f0b8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:26.412701 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039a3a3-9697-45b9-b379-3c9fc0f0b8c7 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:26.424825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 406/811] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:26.425865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039a3a3-9697-45b9-b379-3c9fc0f0b8c7 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:26.483133 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c247-b469-40e4-a6ce-029106ee7674 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:26.527083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6e6bf-8d5a-453b-8804-66cc99fa28a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.528190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6e6bf-8d5a-453b-8804-66cc99fa28a9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.528970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6e6bf-8d5a-453b-8804-66cc99fa28a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.529403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6e6bf-8d5a-453b-8804-66cc99fa28a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.530526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6e6bf-8d5a-453b-8804-66cc99fa28a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:26.531131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6e6bf-8d5a-453b-8804-66cc99fa28a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:26.533309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6e6bf-8d5a-453b-8804-66cc99fa28a9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:26.537583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039a3a3-9697-45b9-b379-3c9fc0f0b8c7 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:26.551226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 406/812] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:26.552436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6e6bf-8d5a-453b-8804-66cc99fa28a9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:26.576244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543df02-1155-456b-9433-174f26292bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.577090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543df02-1155-456b-9433-174f26292bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.577371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543df02-1155-456b-9433-174f26292bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.577647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543df02-1155-456b-9433-174f26292bf3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.578302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543df02-1155-456b-9433-174f26292bf3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:26.580677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543df02-1155-456b-9433-174f26292bf3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-7543df02-1155-456b-9433-174f26292bf3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:26.596129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:26.597194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:26.598647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 407/813] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:26.599471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543df02-1155-456b-9433-174f26292bf3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:26.628436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:26.649785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:26.674078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6e6bf-8d5a-453b-8804-66cc99fa28a9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:26.713380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b680e-4ace-4603-8180-e9cd975bee3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.714536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b680e-4ace-4603-8180-e9cd975bee3d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.715138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b680e-4ace-4603-8180-e9cd975bee3d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.715731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b680e-4ace-4603-8180-e9cd975bee3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.725848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543df02-1155-456b-9433-174f26292bf3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:26.727988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b680e-4ace-4603-8180-e9cd975bee3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:26.729877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b680e-4ace-4603-8180-e9cd975bee3d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-7d6b680e-4ace-4603-8180-e9cd975bee3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:26.749246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:26.750179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:26.751072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:26.751072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:26.751072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:26.751072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:26.754201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 407/814] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:26.755363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b680e-4ace-4603-8180-e9cd975bee3d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:26.764880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175c09-ba1f-49f3-874c-8599a7e59bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.765682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175c09-ba1f-49f3-874c-8599a7e59bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.765956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175c09-ba1f-49f3-874c-8599a7e59bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.766241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175c09-ba1f-49f3-874c-8599a7e59bd4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.775983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c175c09-ba1f-49f3-874c-8599a7e59bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:26.777887 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c175c09-ba1f-49f3-874c-8599a7e59bd4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-9c175c09-ba1f-49f3-874c-8599a7e59bd4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:26.796658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:26.797664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 408/815] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:26.799763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175c09-ba1f-49f3-874c-8599a7e59bd4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:26.894486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b680e-4ace-4603-8180-e9cd975bee3d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:26.914130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:26.917685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:26.922137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175c09-ba1f-49f3-874c-8599a7e59bd4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:26.941268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5388de-8cb8-4b07-a54a-3e455ebb3764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.941939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5388de-8cb8-4b07-a54a-3e455ebb3764 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.942224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5388de-8cb8-4b07-a54a-3e455ebb3764 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.942504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5388de-8cb8-4b07-a54a-3e455ebb3764 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.943227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5388de-8cb8-4b07-a54a-3e455ebb3764 None admin] RBAC: Adding query filter params (name=network) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:26.943545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5388de-8cb8-4b07-a54a-3e455ebb3764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:26.944949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5388de-8cb8-4b07-a54a-3e455ebb3764 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:26.950052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 408/816] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:26.950910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5388de-8cb8-4b07-a54a-3e455ebb3764 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:26.959767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c8738-5656-4178-b9fc-e1e5765ed4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:26.960555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c8738-5656-4178-b9fc-e1e5765ed4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:26.960872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c8738-5656-4178-b9fc-e1e5765ed4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:26.961154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c8738-5656-4178-b9fc-e1e5765ed4e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:26.961885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c8738-5656-4178-b9fc-e1e5765ed4e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:26.962195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c8738-5656-4178-b9fc-e1e5765ed4e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:26.963566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c8738-5656-4178-b9fc-e1e5765ed4e8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:26.977015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 409/817] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:26.977930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c8738-5656-4178-b9fc-e1e5765ed4e8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:27.062462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5388de-8cb8-4b07-a54a-3e455ebb3764 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:27.089608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c8738-5656-4178-b9fc-e1e5765ed4e8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:27.100349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcff5db-45f0-47e3-839a-f7e04bf828de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:27.101057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcff5db-45f0-47e3-839a-f7e04bf828de None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:27.101342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcff5db-45f0-47e3-839a-f7e04bf828de None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:27.101871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcff5db-45f0-47e3-839a-f7e04bf828de None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:27.102813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcff5db-45f0-47e3-839a-f7e04bf828de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:27.103131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcff5db-45f0-47e3-839a-f7e04bf828de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:27.104570 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcff5db-45f0-47e3-839a-f7e04bf828de None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:27.123579 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 409/818] 10.222.0.45 () {56 vars in 1135 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:27.124393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcff5db-45f0-47e3-839a-f7e04bf828de None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:27.127976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4936e-2a4a-4d1f-82c5-a3909a206941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:27.128996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4936e-2a4a-4d1f-82c5-a3909a206941 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:27.129517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4936e-2a4a-4d1f-82c5-a3909a206941 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:27.129992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4936e-2a4a-4d1f-82c5-a3909a206941 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:27.130902 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4936e-2a4a-4d1f-82c5-a3909a206941 None admin] RBAC: Adding query filter params (type=network) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:27.131403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4936e-2a4a-4d1f-82c5-a3909a206941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:27.132976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4936e-2a4a-4d1f-82c5-a3909a206941 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:27.138712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 410/819] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:27.139861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4936e-2a4a-4d1f-82c5-a3909a206941 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:27.162253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:27.167288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:27.232802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcff5db-45f0-47e3-839a-f7e04bf828de None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:27.250580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4936e-2a4a-4d1f-82c5-a3909a206941 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:27.270155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df40c36-316b-4cdb-a413-78b1675eb999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:27.271134 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df40c36-316b-4cdb-a413-78b1675eb999 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:27.271423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df40c36-316b-4cdb-a413-78b1675eb999 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:27.271697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df40c36-316b-4cdb-a413-78b1675eb999 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:27.279387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df40c36-316b-4cdb-a413-78b1675eb999 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:27.280865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df40c36-316b-4cdb-a413-78b1675eb999 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:27.290465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f3faf-00ed-47c6-b15a-5d8dc17901a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:27.291521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3faf-00ed-47c6-b15a-5d8dc17901a5 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:27.292078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3faf-00ed-47c6-b15a-5d8dc17901a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:27.292550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3faf-00ed-47c6-b15a-5d8dc17901a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6df40c36-316b-4cdb-a413-78b1675eb999 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:27.297076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:27.298016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:27.299578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:27.299578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:27.299578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:27.299578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:27.299578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:27.299578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:27.299578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:27.299578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:27.299578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:27.299578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:27.303426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 410/820] 10.222.0.45 () {56 vars in 1117 bytes} [Tue Jul 5 15:51:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:27.304348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df40c36-316b-4cdb-a413-78b1675eb999 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:27.308290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f3faf-00ed-47c6-b15a-5d8dc17901a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:27.309996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f3faf-00ed-47c6-b15a-5d8dc17901a5 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-539f3faf-00ed-47c6-b15a-5d8dc17901a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:27.328590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:27.329507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:27.330423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:27.330423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:27.330423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:27.330423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:27.332856 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 411/821] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:27.333981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f3faf-00ed-47c6-b15a-5d8dc17901a5 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:27.404980 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:27.411950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df40c36-316b-4cdb-a413-78b1675eb999 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:27.415257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:27.444053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f3faf-00ed-47c6-b15a-5d8dc17901a5 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:27.455234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf536419-1917-4637-85b5-e6dcc26eb8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:27.456245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf536419-1917-4637-85b5-e6dcc26eb8f7 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:27.456725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf536419-1917-4637-85b5-e6dcc26eb8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:27.457220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf536419-1917-4637-85b5-e6dcc26eb8f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:27.458161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf536419-1917-4637-85b5-e6dcc26eb8f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:27.459748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf536419-1917-4637-85b5-e6dcc26eb8f7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:27.487157 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3f131-d5c7-44cd-b6f1-b175a53645b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:27.487850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3f131-d5c7-44cd-b6f1-b175a53645b4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:27.488138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3f131-d5c7-44cd-b6f1-b175a53645b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:27.488402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3f131-d5c7-44cd-b6f1-b175a53645b4 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:27.489294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3f131-d5c7-44cd-b6f1-b175a53645b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:27.489603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3f131-d5c7-44cd-b6f1-b175a53645b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:27.491218 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3f131-d5c7-44cd-b6f1-b175a53645b4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:27.511105 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 412/822] 10.222.0.45 () {56 vars in 1149 bytes} [Tue Jul 5 15:51:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:27.513275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3f131-d5c7-44cd-b6f1-b175a53645b4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:27.535425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 411/823] 10.222.0.45 () {60 vars in 1144 bytes} [Tue Jul 5 15:51:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:27.536410 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf536419-1917-4637-85b5-e6dcc26eb8f7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:27.631730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3f131-d5c7-44cd-b6f1-b175a53645b4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:27.678820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:27.680528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa2787-bf44-494e-86bf-0554c87fb748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:27.681592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa2787-bf44-494e-86bf-0554c87fb748 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:27.681971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa2787-bf44-494e-86bf-0554c87fb748 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:27.682338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa2787-bf44-494e-86bf-0554c87fb748 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:27.683174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa2787-bf44-494e-86bf-0554c87fb748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:27.683571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa2787-bf44-494e-86bf-0554c87fb748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:27.685064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa2787-bf44-494e-86bf-0554c87fb748 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:27.699330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 413/824] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:27.706934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa2787-bf44-494e-86bf-0554c87fb748 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:27.775998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf536419-1917-4637-85b5-e6dcc26eb8f7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:27.814435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aae362-89f3-4e94-b03f-0c6d9fa31110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:27.815235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aae362-89f3-4e94-b03f-0c6d9fa31110 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:27.815518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aae362-89f3-4e94-b03f-0c6d9fa31110 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:27.815792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aae362-89f3-4e94-b03f-0c6d9fa31110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:27.828306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aae362-89f3-4e94-b03f-0c6d9fa31110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:27.830392 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aae362-89f3-4e94-b03f-0c6d9fa31110 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:27.833260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa2787-bf44-494e-86bf-0554c87fb748 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-85aae362-89f3-4e94-b03f-0c6d9fa31110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:27.851961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:27.852967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:27.853868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:27.853868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:27.853868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:27.853868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:27.854498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 412/825] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:27.860318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aae362-89f3-4e94-b03f-0c6d9fa31110 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:27.864892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:27.871765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabadd3-bec2-469f-9701-cb3915c8df2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:27.872660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabadd3-bec2-469f-9701-cb3915c8df2c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:27.873026 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabadd3-bec2-469f-9701-cb3915c8df2c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:27.873367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabadd3-bec2-469f-9701-cb3915c8df2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:27.881091 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabadd3-bec2-469f-9701-cb3915c8df2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:27.882654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabadd3-bec2-469f-9701-cb3915c8df2c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-0fabadd3-bec2-469f-9701-cb3915c8df2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:27.897714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:27.898835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:27.900196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:27.900196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:27.900196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:27.900196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:27.900196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:27.900196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:27.900196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:27.900196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:27.901265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 414/826] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:27.907411 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabadd3-bec2-469f-9701-cb3915c8df2c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:27.973221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:27.987481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aae362-89f3-4e94-b03f-0c6d9fa31110 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:28.030263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabadd3-bec2-469f-9701-cb3915c8df2c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:28.035811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bc8b1-4e2f-4610-8bb3-bf4f2f3daca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:28.036484 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bc8b1-4e2f-4610-8bb3-bf4f2f3daca7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.036792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bc8b1-4e2f-4610-8bb3-bf4f2f3daca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.037092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bc8b1-4e2f-4610-8bb3-bf4f2f3daca7 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.038350 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bc8b1-4e2f-4610-8bb3-bf4f2f3daca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:28.038664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bc8b1-4e2f-4610-8bb3-bf4f2f3daca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:28.040637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bc8b1-4e2f-4610-8bb3-bf4f2f3daca7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:28.056948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 413/827] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:28.058219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0448-fba2-4f0c-b406-e0d06c35c8d7 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.058506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0448-fba2-4f0c-b406-e0d06c35c8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.058778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0448-fba2-4f0c-b406-e0d06c35c8d7 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.060183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 414/828] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:28.062985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea0448-fba2-4f0c-b406-e0d06c35c8d7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:28.071503 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8017213-887c-4d91-8493-45255aee5c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:28.072174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8017213-887c-4d91-8493-45255aee5c39 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.072464 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8017213-887c-4d91-8493-45255aee5c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.072746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8017213-887c-4d91-8493-45255aee5c39 None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.073571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8017213-887c-4d91-8493-45255aee5c39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:28.073886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8017213-887c-4d91-8493-45255aee5c39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:28.075307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8017213-887c-4d91-8493-45255aee5c39 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:28.088373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 415/829] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:28.089720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c69699-e861-4a16-bd84-4234938d563f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.090022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c69699-e861-4a16-bd84-4234938d563f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.090299 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c69699-e861-4a16-bd84-4234938d563f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.107228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b4c69699-e861-4a16-bd84-4234938d563f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:28.108416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b4c69699-e861-4a16-bd84-4234938d563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:28.138371 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:28.179407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea0448-fba2-4f0c-b406-e0d06c35c8d7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:28.179989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 416/830] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:28.180916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c69699-e861-4a16-bd84-4234938d563f None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:28.224657 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d0874-190a-4f1c-865c-bfff78a2a47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:28.225242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:28.225363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d0874-190a-4f1c-865c-bfff78a2a47d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.225638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d0874-190a-4f1c-865c-bfff78a2a47d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.225913 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d0874-190a-4f1c-865c-bfff78a2a47d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.233924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d0874-190a-4f1c-865c-bfff78a2a47d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:28.235393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d0874-190a-4f1c-865c-bfff78a2a47d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-dd7d0874-190a-4f1c-865c-bfff78a2a47d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:28.252068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:28.253091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:28.254031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:28.254031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:28.254031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:28.254031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:28.254031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:28.254031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:28.254031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:28.254031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:28.254031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 415/831] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:28.256881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d0874-190a-4f1c-865c-bfff78a2a47d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:28.297036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c69699-e861-4a16-bd84-4234938d563f None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:28.334050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef8907-a27c-41c1-85c8-6e4148549fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:28.334941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef8907-a27c-41c1-85c8-6e4148549fed None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.335227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef8907-a27c-41c1-85c8-6e4148549fed None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.335506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef8907-a27c-41c1-85c8-6e4148549fed None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/8da9cdd70af646928ee0edcadbcb524f/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.337489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef8907-a27c-41c1-85c8-6e4148549fed None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=8da9cdd70af646928ee0edcadbcb524f, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:28.338938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef8907-a27c-41c1-85c8-6e4148549fed None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:28.367425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d0874-190a-4f1c-865c-bfff78a2a47d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:28.391007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:28.401804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d26852-faf4-4c00-a90b-f3cfeacec175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:28.402650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d26852-faf4-4c00-a90b-f3cfeacec175 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.402990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d26852-faf4-4c00-a90b-f3cfeacec175 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.403211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d26852-faf4-4c00-a90b-f3cfeacec175 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.403717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 417/832] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:51:28 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/8da9cdd70af646928ee0edcadbcb524f/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:51:28.403954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d26852-faf4-4c00-a90b-f3cfeacec175 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:28.405457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d26852-faf4-4c00-a90b-f3cfeacec175 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:28.417789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef8907-a27c-41c1-85c8-6e4148549fed None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c7d26852-faf4-4c00-a90b-f3cfeacec175 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:28.418132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:28.419090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:28.431307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 416/833] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:51:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:28.432129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d26852-faf4-4c00-a90b-f3cfeacec175 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:28.549225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:28.549621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d26852-faf4-4c00-a90b-f3cfeacec175 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:28.553013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef8907-a27c-41c1-85c8-6e4148549fed None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:28.590920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db8933-7a06-458e-a999-8967a15019c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:28.591578 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db8933-7a06-458e-a999-8967a15019c2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.591861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db8933-7a06-458e-a999-8967a15019c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.592135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db8933-7a06-458e-a999-8967a15019c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.592951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db8933-7a06-458e-a999-8967a15019c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:28.593269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db8933-7a06-458e-a999-8967a15019c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:28.594696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db8933-7a06-458e-a999-8967a15019c2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:28.598040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a59eb8-440c-44e0-b07a-099f524dfd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:28.598711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a59eb8-440c-44e0-b07a-099f524dfd1c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.598999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a59eb8-440c-44e0-b07a-099f524dfd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.599276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a59eb8-440c-44e0-b07a-099f524dfd1c None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.599991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a59eb8-440c-44e0-b07a-099f524dfd1c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:28.600303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a59eb8-440c-44e0-b07a-099f524dfd1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:28.602122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a59eb8-440c-44e0-b07a-099f524dfd1c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:28.606664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 417/834] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:51:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:28.607688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 418/835] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:28.608022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57135475-2954-4cc8-ac6c-24ac5a7f3fb2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.608317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57135475-2954-4cc8-ac6c-24ac5a7f3fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.608594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57135475-2954-4cc8-ac6c-24ac5a7f3fb2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.616747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 418/836] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:28.616940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db8933-7a06-458e-a999-8967a15019c2 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:28.617856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57135475-2954-4cc8-ac6c-24ac5a7f3fb2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:28.677785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:28.724223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db8933-7a06-458e-a999-8967a15019c2 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:28.729672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57135475-2954-4cc8-ac6c-24ac5a7f3fb2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:28.765833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7ffec-d626-43cb-9103-bb9c28fcc585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:28.766871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7ffec-d626-43cb-9103-bb9c28fcc585 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.767153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7ffec-d626-43cb-9103-bb9c28fcc585 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.767438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7ffec-d626-43cb-9103-bb9c28fcc585 None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/7245effe463b45dfa946ca2eda2e0c98/roles/ac08a857de8b484fb5dfc65cd4cdc044` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.768042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968c5f0-6974-4674-8017-ef72409ec5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:28.768700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968c5f0-6974-4674-8017-ef72409ec5eb None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.769007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968c5f0-6974-4674-8017-ef72409ec5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.769285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968c5f0-6974-4674-8017-ef72409ec5eb None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.769999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968c5f0-6974-4674-8017-ef72409ec5eb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:28.770304 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968c5f0-6974-4674-8017-ef72409ec5eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:28.771661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968c5f0-6974-4674-8017-ef72409ec5eb None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:28.780800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 419/837] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:51:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:28.780800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f9294-9d3f-4995-b6f0-68ade68b0a27 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:28.780800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f9294-9d3f-4995-b6f0-68ade68b0a27 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:28.780800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f9294-9d3f-4995-b6f0-68ade68b0a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:28.795501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7ffec-d626-43cb-9103-bb9c28fcc585 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=7245effe463b45dfa946ca2eda2e0c98, role_id=ac08a857de8b484fb5dfc65cd4cdc044)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:28.797060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7ffec-d626-43cb-9103-bb9c28fcc585 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:28.805989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-6f0f9294-9d3f-4995-b6f0-68ade68b0a27 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:28.807134 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-6f0f9294-9d3f-4995-b6f0-68ade68b0a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:28.826266 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:28.882837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 420/838] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:28.894297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 419/839] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:51:28 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/7245effe463b45dfa946ca2eda2e0c98/roles/ac08a857de8b484fb5dfc65cd4cdc044 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:51:28.914178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f9294-9d3f-4995-b6f0-68ade68b0a27 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:29.031364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f9294-9d3f-4995-b6f0-68ade68b0a27 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:29.070136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea242-548e-48b8-85dd-54e892f24236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:29.070880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea242-548e-48b8-85dd-54e892f24236 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:29.071164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea242-548e-48b8-85dd-54e892f24236 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:29.071436 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea242-548e-48b8-85dd-54e892f24236 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:29.072089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dea242-548e-48b8-85dd-54e892f24236 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:29.073489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dea242-548e-48b8-85dd-54e892f24236 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-e8dea242-548e-48b8-85dd-54e892f24236 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:29.086721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:29.087694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:29.088527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 421/840] 10.222.0.45 () {56 vars in 1132 bytes} [Tue Jul 5 15:51:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:29.094013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7ffec-d626-43cb-9103-bb9c28fcc585 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:29.201502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7ffec-d626-43cb-9103-bb9c28fcc585 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:29.203066 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:29.237589 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad8b80-51d2-4300-aa01-138a6f1c6c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:29.238262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad8b80-51d2-4300-aa01-138a6f1c6c9f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:29.238548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad8b80-51d2-4300-aa01-138a6f1c6c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:29.238827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad8b80-51d2-4300-aa01-138a6f1c6c9f None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:29.239539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad8b80-51d2-4300-aa01-138a6f1c6c9f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:29.239849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad8b80-51d2-4300-aa01-138a6f1c6c9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:29.241249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad8b80-51d2-4300-aa01-138a6f1c6c9f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:29.246627 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 420/841] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:29.252204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea242-548e-48b8-85dd-54e892f24236 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:29.276864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:29.369733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea242-548e-48b8-85dd-54e892f24236 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:29.406590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01628424-9459-43fb-81f9-9ad34b7479cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:29.407708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01628424-9459-43fb-81f9-9ad34b7479cf None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:29.408200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01628424-9459-43fb-81f9-9ad34b7479cf None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:29.408679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01628424-9459-43fb-81f9-9ad34b7479cf None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:29.409621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01628424-9459-43fb-81f9-9ad34b7479cf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:29.410131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01628424-9459-43fb-81f9-9ad34b7479cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:29.411904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01628424-9459-43fb-81f9-9ad34b7479cf None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:29.417457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 422/842] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:29 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:30.483930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23233a9c-9783-4509-9bd6-11d4eefaad49 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:30.484927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23233a9c-9783-4509-9bd6-11d4eefaad49 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:30.485463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23233a9c-9783-4509-9bd6-11d4eefaad49 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:30.487345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 421/843] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:30.492544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7208965-b996-4e83-95ce-b933fd9fb862 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:30.492981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7208965-b996-4e83-95ce-b933fd9fb862 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:30.493152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7208965-b996-4e83-95ce-b933fd9fb862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:30.510024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e7208965-b996-4e83-95ce-b933fd9fb862 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:30.511230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e7208965-b996-4e83-95ce-b933fd9fb862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:30.588958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 423/844] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:30.607097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23233a9c-9783-4509-9bd6-11d4eefaad49 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:30.712156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23233a9c-9783-4509-9bd6-11d4eefaad49 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:30.747910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db58fd-37b1-4116-9b73-0bd2199f27d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:30.748700 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db58fd-37b1-4116-9b73-0bd2199f27d8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:30.749006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db58fd-37b1-4116-9b73-0bd2199f27d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:30.749283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db58fd-37b1-4116-9b73-0bd2199f27d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:30.759981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db58fd-37b1-4116-9b73-0bd2199f27d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a3db58fd-37b1-4116-9b73-0bd2199f27d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:30.767725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:30.768655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:30.769529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 422/845] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:30.774936 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7208965-b996-4e83-95ce-b933fd9fb862 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:30.809150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:30.881308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7208965-b996-4e83-95ce-b933fd9fb862 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:30.921573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66dc65-02b7-430b-8933-ecad57a6bebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:30.922738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66dc65-02b7-430b-8933-ecad57a6bebf None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:30.923800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66dc65-02b7-430b-8933-ecad57a6bebf None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:30.924256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66dc65-02b7-430b-8933-ecad57a6bebf None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:30.924482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e7080-3329-49ea-a58a-b6aec48dd396 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:30.924796 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e7080-3329-49ea-a58a-b6aec48dd396 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:30.925096 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e7080-3329-49ea-a58a-b6aec48dd396 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:30.925772 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66dc65-02b7-430b-8933-ecad57a6bebf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:30.926248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66dc65-02b7-430b-8933-ecad57a6bebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:30.926866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 423/846] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:30.928152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66dc65-02b7-430b-8933-ecad57a6bebf None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:30.931855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39514bd0-c9fa-4756-b9cf-58d7ba20037e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:30.932143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39514bd0-c9fa-4756-b9cf-58d7ba20037e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:30.932588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39514bd0-c9fa-4756-b9cf-58d7ba20037e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:30.942676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 424/847] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:30.944217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66dc65-02b7-430b-8933-ecad57a6bebf None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:30.949776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-39514bd0-c9fa-4756-b9cf-58d7ba20037e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:30.951099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-39514bd0-c9fa-4756-b9cf-58d7ba20037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:30.974237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:31.026341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 424/848] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:31.035849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:31.051176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39514bd0-c9fa-4756-b9cf-58d7ba20037e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:31.069729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66dc65-02b7-430b-8933-ecad57a6bebf None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:31.108111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857a2c3-5dd1-46b8-b6ed-27918914eea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:31.108950 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857a2c3-5dd1-46b8-b6ed-27918914eea3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.109238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857a2c3-5dd1-46b8-b6ed-27918914eea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.109519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857a2c3-5dd1-46b8-b6ed-27918914eea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.121717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857a2c3-5dd1-46b8-b6ed-27918914eea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:31.123369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857a2c3-5dd1-46b8-b6ed-27918914eea3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-9857a2c3-5dd1-46b8-b6ed-27918914eea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:31.141853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:31.142947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:31.143829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:31.143829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:31.143829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:31.143829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:31.143829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 425/849] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:31.144494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4fa69-845b-454c-965d-8363a27b42d8 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.144494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4fa69-845b-454c-965d-8363a27b42d8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.144783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4fa69-845b-454c-965d-8363a27b42d8 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.148912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 426/850] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:31.148912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4fa69-845b-454c-965d-8363a27b42d8 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:31.163418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39514bd0-c9fa-4756-b9cf-58d7ba20037e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:31.202177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee9ff6-d52c-42e0-9e62-a21286d6602c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:31.202849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee9ff6-d52c-42e0-9e62-a21286d6602c None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.203130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee9ff6-d52c-42e0-9e62-a21286d6602c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.203401 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee9ff6-d52c-42e0-9e62-a21286d6602c None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.204128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee9ff6-d52c-42e0-9e62-a21286d6602c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:31.206088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee9ff6-d52c-42e0-9e62-a21286d6602c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:31.261349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4fa69-845b-454c-965d-8363a27b42d8 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:31.277186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 425/851] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:51:31 2022] POST /identity/v3/services => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:51:31.279179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b3b71-8392-4d5d-aac6-3c828b03c380 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.279679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b3b71-8392-4d5d-aac6-3c828b03c380 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.280506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b3b71-8392-4d5d-aac6-3c828b03c380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.288564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:31.307399 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ea4b3b71-8392-4d5d-aac6-3c828b03c380 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:31.308673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ea4b3b71-8392-4d5d-aac6-3c828b03c380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:31.317963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab43f91-e39e-4e99-8113-e67efb9d864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:31.319604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab43f91-e39e-4e99-8113-e67efb9d864b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.319604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab43f91-e39e-4e99-8113-e67efb9d864b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.319604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab43f91-e39e-4e99-8113-e67efb9d864b None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.320492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab43f91-e39e-4e99-8113-e67efb9d864b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:31.320899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab43f91-e39e-4e99-8113-e67efb9d864b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:31.322549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:31.322731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab43f91-e39e-4e99-8113-e67efb9d864b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:31.342773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 427/852] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:31.344395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab43f91-e39e-4e99-8113-e67efb9d864b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:31.406825 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 426/853] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:31.430459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b3b71-8392-4d5d-aac6-3c828b03c380 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:31.469699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab43f91-e39e-4e99-8113-e67efb9d864b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:31.512582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0eb34a-044a-4b9d-826b-ff3ec404bcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:31.517554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb34a-044a-4b9d-826b-ff3ec404bcfd None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.517554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb34a-044a-4b9d-826b-ff3ec404bcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.517554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb34a-044a-4b9d-826b-ff3ec404bcfd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.531928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0eb34a-044a-4b9d-826b-ff3ec404bcfd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:31.534100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0eb34a-044a-4b9d-826b-ff3ec404bcfd None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-ff0eb34a-044a-4b9d-826b-ff3ec404bcfd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:31.548826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:31.549784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:31.550714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:31.550714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:31.550714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:31.550714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:31.550714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:31.550714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:31.550714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:31.550714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:31.550714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:51:31.550714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:31.551238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b3b71-8392-4d5d-aac6-3c828b03c380 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:31.551632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 428/854] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:31.551632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4b824-be39-49d8-811a-fe16a87a9f24 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.552516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4b824-be39-49d8-811a-fe16a87a9f24 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.552516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4b824-be39-49d8-811a-fe16a87a9f24 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.553573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 429/855] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:31.557661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4b824-be39-49d8-811a-fe16a87a9f24 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:31.588399 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7e312-de80-44c8-99e5-3744cfa21db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:31.589236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7e312-de80-44c8-99e5-3744cfa21db7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.589504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7e312-de80-44c8-99e5-3744cfa21db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.589779 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7e312-de80-44c8-99e5-3744cfa21db7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.601393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7e312-de80-44c8-99e5-3744cfa21db7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:31.606048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-15a7e312-de80-44c8-99e5-3744cfa21db7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:31.609188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:31.610128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:31.611007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 427/856] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:51:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:31.611713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d4ce6-6612-497b-83f7-43f3f27de608 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.612116 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d4ce6-6612-497b-83f7-43f3f27de608 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.612204 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d4ce6-6612-497b-83f7-43f3f27de608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.625681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:31.632318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-449d4ce6-6612-497b-83f7-43f3f27de608 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:31.634037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-449d4ce6-6612-497b-83f7-43f3f27de608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:31.674205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4b824-be39-49d8-811a-fe16a87a9f24 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:31.704016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 428/857] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:31.705960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d4ce6-6612-497b-83f7-43f3f27de608 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:31.715557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d104e2-a1ea-428c-a8be-dec3bb18d280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:31.716217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d104e2-a1ea-428c-a8be-dec3bb18d280 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.716497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d104e2-a1ea-428c-a8be-dec3bb18d280 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.716793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d104e2-a1ea-428c-a8be-dec3bb18d280 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.718449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d104e2-a1ea-428c-a8be-dec3bb18d280 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:31.718759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d104e2-a1ea-428c-a8be-dec3bb18d280 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:31.720197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d104e2-a1ea-428c-a8be-dec3bb18d280 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:31.738347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 430/858] 10.222.0.45 () {56 vars in 1135 bytes} [Tue Jul 5 15:51:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:31.739233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d104e2-a1ea-428c-a8be-dec3bb18d280 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:31.827104 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d4ce6-6612-497b-83f7-43f3f27de608 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:31.837339 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:31.841853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d104e2-a1ea-428c-a8be-dec3bb18d280 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:31.865884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e6334-1f21-4822-847d-86d9f17f6b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:31.866559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e6334-1f21-4822-847d-86d9f17f6b39 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.866864 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e6334-1f21-4822-847d-86d9f17f6b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.867112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e6334-1f21-4822-847d-86d9f17f6b39 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.867871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e6334-1f21-4822-847d-86d9f17f6b39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:31.868211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e6334-1f21-4822-847d-86d9f17f6b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:31.869594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e6334-1f21-4822-847d-86d9f17f6b39 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:31.872797 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:31.885303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 429/859] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:51:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:31.885303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e6334-1f21-4822-847d-86d9f17f6b39 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:31.892854 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48a539-7962-42d2-a1d8-4f10613f7e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:31.893552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48a539-7962-42d2-a1d8-4f10613f7e00 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:31.893831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48a539-7962-42d2-a1d8-4f10613f7e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:31.894166 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48a539-7962-42d2-a1d8-4f10613f7e00 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:31.894927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48a539-7962-42d2-a1d8-4f10613f7e00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:31.896376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48a539-7962-42d2-a1d8-4f10613f7e00 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:31.986194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e6334-1f21-4822-847d-86d9f17f6b39 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:32.019463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbc79f-ac38-4515-88f6-ce20f7ff8156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:32.020110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbc79f-ac38-4515-88f6-ce20f7ff8156 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:32.020394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbc79f-ac38-4515-88f6-ce20f7ff8156 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:32.020669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbc79f-ac38-4515-88f6-ce20f7ff8156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:32.029109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 431/860] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:51:31 2022] POST /identity/v3/services => generated 269 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:32.029109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48a539-7962-42d2-a1d8-4f10613f7e00 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:32.125156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:32.133054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbc79f-ac38-4515-88f6-ce20f7ff8156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:32.134791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbc79f-ac38-4515-88f6-ce20f7ff8156 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:32.161752 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48a539-7962-42d2-a1d8-4f10613f7e00 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:32.213573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ecd8e-834c-4d64-ad51-0fc9e9da970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:32.214014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ecd8e-834c-4d64-ad51-0fc9e9da970e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:32.214294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ecd8e-834c-4d64-ad51-0fc9e9da970e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:32.214569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ecd8e-834c-4d64-ad51-0fc9e9da970e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:32.227992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ecd8e-834c-4d64-ad51-0fc9e9da970e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:32.229476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ecd8e-834c-4d64-ad51-0fc9e9da970e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-c01ecd8e-834c-4d64-ad51-0fc9e9da970e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:32.248298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:32.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:32.250121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:32.250121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:32.250121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:32.250121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:32.250121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 432/861] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:32.258848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ecd8e-834c-4d64-ad51-0fc9e9da970e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:32.266668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 430/862] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:31 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:51:32.268203 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:32.275299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbc79f-ac38-4515-88f6-ce20f7ff8156 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:32.367631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ecd8e-834c-4d64-ad51-0fc9e9da970e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:32.403000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7b31b-561a-4837-b529-ceed874e0f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:32.403871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7b31b-561a-4837-b529-ceed874e0f3d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:32.404533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:32.404654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7b31b-561a-4837-b529-ceed874e0f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:32.406003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7b31b-561a-4837-b529-ceed874e0f3d None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:32.406822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7b31b-561a-4837-b529-ceed874e0f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:32.408561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7b31b-561a-4837-b529-ceed874e0f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:32.410060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7b31b-561a-4837-b529-ceed874e0f3d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:32.410543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbc79f-ac38-4515-88f6-ce20f7ff8156 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:32.424812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 433/863] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:32.430212 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7b31b-561a-4837-b529-ceed874e0f3d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:32.456585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03794e8-adc5-4330-8e39-b4638bd5001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:32.457808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03794e8-adc5-4330-8e39-b4638bd5001a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:32.458298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03794e8-adc5-4330-8e39-b4638bd5001a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:32.458772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03794e8-adc5-4330-8e39-b4638bd5001a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:32.471182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03794e8-adc5-4330-8e39-b4638bd5001a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:32.473045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03794e8-adc5-4330-8e39-b4638bd5001a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a03794e8-adc5-4330-8e39-b4638bd5001a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:32.496040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:32.497047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:32.497957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:32.497957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:32.497957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:32.497957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:32.500345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 431/864] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:32.506408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03794e8-adc5-4330-8e39-b4638bd5001a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:32.515046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:32.558806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7b31b-561a-4837-b529-ceed874e0f3d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:32.601272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed667e-ce47-485b-b4ac-9fa415693514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:32.602359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed667e-ce47-485b-b4ac-9fa415693514 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:32.602642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed667e-ce47-485b-b4ac-9fa415693514 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:32.602917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed667e-ce47-485b-b4ac-9fa415693514 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:32.613013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed667e-ce47-485b-b4ac-9fa415693514 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:32.614712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed667e-ce47-485b-b4ac-9fa415693514 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:32.618856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03794e8-adc5-4330-8e39-b4638bd5001a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-d3ed667e-ce47-485b-b4ac-9fa415693514 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:32.632193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:32.633450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:32.634470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:32.634470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:32.634470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:32.634470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:32.634470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:32.634470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:32.634470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:32.634470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:32.634470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:32.634470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:32.636920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 434/865] 10.222.0.45 () {56 vars in 1117 bytes} [Tue Jul 5 15:51:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:32.639505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed667e-ce47-485b-b4ac-9fa415693514 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:32.662152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d56d84-e609-4a24-adcb-9b3043d3985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:32.663324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d56d84-e609-4a24-adcb-9b3043d3985e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:32.663812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d56d84-e609-4a24-adcb-9b3043d3985e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:32.664356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d56d84-e609-4a24-adcb-9b3043d3985e None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:32.665381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d56d84-e609-4a24-adcb-9b3043d3985e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:32.665893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d56d84-e609-4a24-adcb-9b3043d3985e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:32.667571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d56d84-e609-4a24-adcb-9b3043d3985e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:32.685246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 432/866] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:32.686384 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d56d84-e609-4a24-adcb-9b3043d3985e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:32.700140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:32.760746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed667e-ce47-485b-b4ac-9fa415693514 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:32.793932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:32.801062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d56d84-e609-4a24-adcb-9b3043d3985e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:32.806195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cb1f1-8741-405c-9708-198a187ea1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:32.807067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cb1f1-8741-405c-9708-198a187ea1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:32.807350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cb1f1-8741-405c-9708-198a187ea1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:32.807627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cb1f1-8741-405c-9708-198a187ea1c3 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:32.811825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cb1f1-8741-405c-9708-198a187ea1c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:32.811825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cb1f1-8741-405c-9708-198a187ea1c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:32.811825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cb1f1-8741-405c-9708-198a187ea1c3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:32.840925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 435/867] 10.222.0.45 () {56 vars in 1149 bytes} [Tue Jul 5 15:51:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:32.846621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cb1f1-8741-405c-9708-198a187ea1c3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:32.851868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fff524-2336-4733-974a-5542b1417010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:32.853010 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fff524-2336-4733-974a-5542b1417010 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:32.853457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fff524-2336-4733-974a-5542b1417010 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:32.853891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fff524-2336-4733-974a-5542b1417010 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:32.862093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fff524-2336-4733-974a-5542b1417010 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:32.863976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fff524-2336-4733-974a-5542b1417010 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e3fff524-2336-4733-974a-5542b1417010 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:32.879918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:32.887100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:32.888021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:32.888021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:32.888021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:32.888021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:32.888021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:32.888021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:32.888021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:32.888021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:32.888659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 433/868] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:32.889682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fff524-2336-4733-974a-5542b1417010 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:32.970731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:32.975858 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cb1f1-8741-405c-9708-198a187ea1c3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:33.005355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fff524-2336-4733-974a-5542b1417010 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:33.016630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d9bd9-cebb-41f1-80d7-3f59139156b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:33.018103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d9bd9-cebb-41f1-80d7-3f59139156b7 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:33.018566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d9bd9-cebb-41f1-80d7-3f59139156b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:33.019099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d9bd9-cebb-41f1-80d7-3f59139156b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:33.047655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcca12d-3f7e-4b95-8868-409e503e0cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:33.048341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcca12d-3f7e-4b95-8868-409e503e0cdb None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:33.048625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcca12d-3f7e-4b95-8868-409e503e0cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:33.048924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcca12d-3f7e-4b95-8868-409e503e0cdb None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:33.049869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcca12d-3f7e-4b95-8868-409e503e0cdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:33.050178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcca12d-3f7e-4b95-8868-409e503e0cdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:33.052230 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcca12d-3f7e-4b95-8868-409e503e0cdb None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:33.067438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 434/869] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:33.073081 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcca12d-3f7e-4b95-8868-409e503e0cdb None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:33.078148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:33.096786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d9bd9-cebb-41f1-80d7-3f59139156b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:33.098471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d9bd9-cebb-41f1-80d7-3f59139156b7 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:33.208017 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcca12d-3f7e-4b95-8868-409e503e0cdb None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:33.216800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 436/870] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:32 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:51:33.222133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d9bd9-cebb-41f1-80d7-3f59139156b7 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:33.249225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:33.259979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137ee56-bc1c-43f8-9c03-d7a49cfb982f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:33.261056 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137ee56-bc1c-43f8-9c03-d7a49cfb982f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:33.261601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137ee56-bc1c-43f8-9c03-d7a49cfb982f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:33.262046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137ee56-bc1c-43f8-9c03-d7a49cfb982f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:33.263081 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137ee56-bc1c-43f8-9c03-d7a49cfb982f None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=8da9cdd70af646928ee0edcadbcb524f) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:33.263612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137ee56-bc1c-43f8-9c03-d7a49cfb982f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:33.265329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137ee56-bc1c-43f8-9c03-d7a49cfb982f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:33.289022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 435/871] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:33 2022] GET /identity/v3/role_assignments?user.id=8da9cdd70af646928ee0edcadbcb524f&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:33.326930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:33.345461 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d9bd9-cebb-41f1-80d7-3f59139156b7 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:33.384453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9db7b-5994-429f-8350-022842564bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:33.385275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9db7b-5994-429f-8350-022842564bac None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:33.385553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9db7b-5994-429f-8350-022842564bac None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:33.385821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9db7b-5994-429f-8350-022842564bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:33.398120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9db7b-5994-429f-8350-022842564bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:33.399588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9db7b-5994-429f-8350-022842564bac None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:33.407961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796d995-1af1-43d5-8902-edfba9aeae71 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:33.408261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796d995-1af1-43d5-8902-edfba9aeae71 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:33.408536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796d995-1af1-43d5-8902-edfba9aeae71 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:33.410280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 436/872] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:51:33.415699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b776d-4435-4415-9f77-e3ae18bd3e9f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:33.415983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b776d-4435-4415-9f77-e3ae18bd3e9f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:33.416270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b776d-4435-4415-9f77-e3ae18bd3e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-f4b9db7b-5994-429f-8350-022842564bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:33.419004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:33.419872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:33.420744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:33.420744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:33.420744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:33.420744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:33.420744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 437/873] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:33.426124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9db7b-5994-429f-8350-022842564bac None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:33.437865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bc8b776d-4435-4415-9f77-e3ae18bd3e9f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:33.439644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bc8b776d-4435-4415-9f77-e3ae18bd3e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:33.531376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 437/874] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:33.556169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b776d-4435-4415-9f77-e3ae18bd3e9f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:33.556901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9db7b-5994-429f-8350-022842564bac None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:33.603893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee83f5-cf4c-4136-8506-313b33c8bd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:33.605467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee83f5-cf4c-4136-8506-313b33c8bd1b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:33.605986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee83f5-cf4c-4136-8506-313b33c8bd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:33.606685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee83f5-cf4c-4136-8506-313b33c8bd1b None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:33.607601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee83f5-cf4c-4136-8506-313b33c8bd1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:33.608211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee83f5-cf4c-4136-8506-313b33c8bd1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:33.610073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee83f5-cf4c-4136-8506-313b33c8bd1b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:33.613727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:33.625966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 438/875] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:33.632359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee83f5-cf4c-4136-8506-313b33c8bd1b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:33.673598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b776d-4435-4415-9f77-e3ae18bd3e9f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:33.704070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:33.717283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0647f19-b6ee-4696-9637-73fa9a7167b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:33.718177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647f19-b6ee-4696-9637-73fa9a7167b2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:33.718532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647f19-b6ee-4696-9637-73fa9a7167b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:33.718877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647f19-b6ee-4696-9637-73fa9a7167b2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:33.720006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0647f19-b6ee-4696-9637-73fa9a7167b2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:33.721985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0647f19-b6ee-4696-9637-73fa9a7167b2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-a0647f19-b6ee-4696-9637-73fa9a7167b2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:33.738277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:33.739471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:51:33.740544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:33.744808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 438/876] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:51:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:33.750151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0647f19-b6ee-4696-9637-73fa9a7167b2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:33.755500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee83f5-cf4c-4136-8506-313b33c8bd1b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:33.801060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cb19e-3e82-4eb4-bd26-b9266c805651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:33.802116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cb19e-3e82-4eb4-bd26-b9266c805651 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:33.802550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cb19e-3e82-4eb4-bd26-b9266c805651 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:33.802976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cb19e-3e82-4eb4-bd26-b9266c805651 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:33.812674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cb19e-3e82-4eb4-bd26-b9266c805651 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:33.815624 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cb19e-3e82-4eb4-bd26-b9266c805651 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-802cb19e-3e82-4eb4-bd26-b9266c805651 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:33.835837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:33.836983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:33.838072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:33.838072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:33.838072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:33.838072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:33.838072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:33.838072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:33.838072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:33.838072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:33.846186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 439/877] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:33.847771 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cb19e-3e82-4eb4-bd26-b9266c805651 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:33.875470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0647f19-b6ee-4696-9637-73fa9a7167b2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:33.912448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb301f5-321f-44e2-bc6b-b249cf68f388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:33.913384 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb301f5-321f-44e2-bc6b-b249cf68f388 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:33.913769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb301f5-321f-44e2-bc6b-b249cf68f388 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:33.914144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb301f5-321f-44e2-bc6b-b249cf68f388 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:33.914961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb301f5-321f-44e2-bc6b-b249cf68f388 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:33.915371 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb301f5-321f-44e2-bc6b-b249cf68f388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:33.916903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb301f5-321f-44e2-bc6b-b249cf68f388 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:33.926949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 439/878] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:51:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:33.927875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:33.928608 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb301f5-321f-44e2-bc6b-b249cf68f388 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:33.988853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cb19e-3e82-4eb4-bd26-b9266c805651 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:33.993574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:34.026882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8280b-8bff-4c0f-82d8-dd7334e50c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:34.028005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8280b-8bff-4c0f-82d8-dd7334e50c6a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:34.028583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8280b-8bff-4c0f-82d8-dd7334e50c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:34.029112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8280b-8bff-4c0f-82d8-dd7334e50c6a None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:34.030099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8280b-8bff-4c0f-82d8-dd7334e50c6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:34.030611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8280b-8bff-4c0f-82d8-dd7334e50c6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:34.032348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8280b-8bff-4c0f-82d8-dd7334e50c6a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:34.044491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb301f5-321f-44e2-bc6b-b249cf68f388 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:34.047157 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 440/879] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:34.056531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8280b-8bff-4c0f-82d8-dd7334e50c6a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:34.084747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf57d59-e7b4-4ae0-b013-7a218160ac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:34.085355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf57d59-e7b4-4ae0-b013-7a218160ac5c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:34.085647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf57d59-e7b4-4ae0-b013-7a218160ac5c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:34.085926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf57d59-e7b4-4ae0-b013-7a218160ac5c None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:34.086757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf57d59-e7b4-4ae0-b013-7a218160ac5c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:34.087071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf57d59-e7b4-4ae0-b013-7a218160ac5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:34.088568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf57d59-e7b4-4ae0-b013-7a218160ac5c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:34.094127 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 440/880] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:51:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:34.100397 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf57d59-e7b4-4ae0-b013-7a218160ac5c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:34.169694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8280b-8bff-4c0f-82d8-dd7334e50c6a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:34.188439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:34.207658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a77f23-27ea-414b-8a31-35919dace8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:34.208646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a77f23-27ea-414b-8a31-35919dace8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:34.208965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a77f23-27ea-414b-8a31-35919dace8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:34.216614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a77f23-27ea-414b-8a31-35919dace8b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:34.216614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a77f23-27ea-414b-8a31-35919dace8b9 None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=7245effe463b45dfa946ca2eda2e0c98) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:34.216614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a77f23-27ea-414b-8a31-35919dace8b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:34.216614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a77f23-27ea-414b-8a31-35919dace8b9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:34.216987 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf57d59-e7b4-4ae0-b013-7a218160ac5c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:34.224791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 441/881] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:34 2022] GET /identity/v3/role_assignments?user.id=7245effe463b45dfa946ca2eda2e0c98&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:34.224791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44add61-0bfa-42b8-8273-dd8d0cf74b69 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:34.224791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44add61-0bfa-42b8-8273-dd8d0cf74b69 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:34.224791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44add61-0bfa-42b8-8273-dd8d0cf74b69 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:34.224791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 442/882] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:34.230354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a811ba-89d1-49dc-90e0-82c1485b31cb None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:34.230748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a811ba-89d1-49dc-90e0-82c1485b31cb None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:34.231122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a811ba-89d1-49dc-90e0-82c1485b31cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:34.237340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:34.252749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-46a811ba-89d1-49dc-90e0-82c1485b31cb None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:34.254738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-46a811ba-89d1-49dc-90e0-82c1485b31cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:34.260188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92781039-3263-4afd-b89e-7443b9280117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:34.261173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92781039-3263-4afd-b89e-7443b9280117 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:34.261660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92781039-3263-4afd-b89e-7443b9280117 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:34.262090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92781039-3263-4afd-b89e-7443b9280117 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:34.263125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92781039-3263-4afd-b89e-7443b9280117 None admin] RBAC: Adding query filter params (interface=public, service_id=d37b5c0f6443464aab783ceab1655b3c, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:34.263594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92781039-3263-4afd-b89e-7443b9280117 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:34.265433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92781039-3263-4afd-b89e-7443b9280117 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:34.282120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 441/883] 10.222.0.45 () {56 vars in 1251 bytes} [Tue Jul 5 15:51:34 2022] GET /identity/v3/endpoints?service_id=d37b5c0f6443464aab783ceab1655b3c&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:34.290055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92781039-3263-4afd-b89e-7443b9280117 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:34.357774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 443/884] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:34.379620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a811ba-89d1-49dc-90e0-82c1485b31cb None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:34.420034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92781039-3263-4afd-b89e-7443b9280117 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:34.472177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f43ca-71be-4b4b-845b-399c026e87af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:34.473001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f43ca-71be-4b4b-845b-399c026e87af None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:34.473348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f43ca-71be-4b4b-845b-399c026e87af None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:34.473683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f43ca-71be-4b4b-845b-399c026e87af None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:34.474439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f43ca-71be-4b4b-845b-399c026e87af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:34.475890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f43ca-71be-4b4b-845b-399c026e87af None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:34.484277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 442/885] 10.222.0.45 () {56 vars in 1087 bytes} [Tue Jul 5 15:51:34 2022] GET /identity/v3/services => generated 1268 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:51:34.516541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a811ba-89d1-49dc-90e0-82c1485b31cb None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:34.530071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:34.559832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f76c-0e90-4d59-828c-7e195119c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:34.560671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f76c-0e90-4d59-828c-7e195119c5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:34.561022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f76c-0e90-4d59-828c-7e195119c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:34.561302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843f76c-0e90-4d59-828c-7e195119c5e4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:34.562056 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843f76c-0e90-4d59-828c-7e195119c5e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:34.563474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843f76c-0e90-4d59-828c-7e195119c5e4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-4843f76c-0e90-4d59-828c-7e195119c5e4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:34.577683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:34.578704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:51:34.580083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:34.581498 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 444/886] 10.222.0.45 () {56 vars in 1132 bytes} [Tue Jul 5 15:51:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:34.585843 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f43ca-71be-4b4b-845b-399c026e87af None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:34.637867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:34.698073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f43ca-71be-4b4b-845b-399c026e87af None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:34.733497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e8b82-e8dc-440e-a108-0d338511f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:34.734157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e8b82-e8dc-440e-a108-0d338511f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:34.734442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e8b82-e8dc-440e-a108-0d338511f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:34.734716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e8b82-e8dc-440e-a108-0d338511f0f4 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:34.735423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e8b82-e8dc-440e-a108-0d338511f0f4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:34.735760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e8b82-e8dc-440e-a108-0d338511f0f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:34.737159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e8b82-e8dc-440e-a108-0d338511f0f4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:34.742191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 443/887] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:34.747676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f76c-0e90-4d59-828c-7e195119c5e4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:34.854312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843f76c-0e90-4d59-828c-7e195119c5e4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:34.893822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9143a7-7e3b-4f8b-9c4a-d40a63ee9120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:34.894479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9143a7-7e3b-4f8b-9c4a-d40a63ee9120 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:34.894766 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9143a7-7e3b-4f8b-9c4a-d40a63ee9120 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:34.895044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9143a7-7e3b-4f8b-9c4a-d40a63ee9120 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:34.895760 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9143a7-7e3b-4f8b-9c4a-d40a63ee9120 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:34.896071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9143a7-7e3b-4f8b-9c4a-d40a63ee9120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:34.897456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9143a7-7e3b-4f8b-9c4a-d40a63ee9120 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:34.903469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 445/888] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:34.909752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e8b82-e8dc-440e-a108-0d338511f0f4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:35.015112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e8b82-e8dc-440e-a108-0d338511f0f4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:35.025137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:35.055150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216f180-8fb3-4100-b3ab-120b3118be4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:35.055815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216f180-8fb3-4100-b3ab-120b3118be4c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:35.056101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216f180-8fb3-4100-b3ab-120b3118be4c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:35.056378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216f180-8fb3-4100-b3ab-120b3118be4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:35.057212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216f180-8fb3-4100-b3ab-120b3118be4c None admin] RBAC: Adding query filter params (interface=public, service_id=7936a000d5a94d798dc35f076f175eff, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:35.057521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216f180-8fb3-4100-b3ab-120b3118be4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:35.058911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216f180-8fb3-4100-b3ab-120b3118be4c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:35.064943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 444/889] 10.222.0.45 () {56 vars in 1251 bytes} [Tue Jul 5 15:51:34 2022] GET /identity/v3/endpoints?service_id=7936a000d5a94d798dc35f076f175eff&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:35.069963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9143a7-7e3b-4f8b-9c4a-d40a63ee9120 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:35.117133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:35.175169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9143a7-7e3b-4f8b-9c4a-d40a63ee9120 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:35.209687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565a10e-4c70-4a2d-95ca-ccf9b1d74ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:35.210355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a10e-4c70-4a2d-95ca-ccf9b1d74ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:35.210706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a10e-4c70-4a2d-95ca-ccf9b1d74ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:35.210981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a10e-4c70-4a2d-95ca-ccf9b1d74ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:35.211678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565a10e-4c70-4a2d-95ca-ccf9b1d74ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:35.213135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565a10e-4c70-4a2d-95ca-ccf9b1d74ef1 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:35.219189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 446/890] 10.222.0.45 () {56 vars in 1087 bytes} [Tue Jul 5 15:51:35 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:51:35.550140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9522-4f8a-487b-8629-f8f6e694042d None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:35.550335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9522-4f8a-487b-8629-f8f6e694042d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:35.550620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9522-4f8a-487b-8629-f8f6e694042d None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:35.552263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 445/891] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:35.557871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e43d6b-8182-43c6-88f2-caefdbe49851 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:35.558166 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e43d6b-8182-43c6-88f2-caefdbe49851 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:35.558445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e43d6b-8182-43c6-88f2-caefdbe49851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:35.576227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b5e43d6b-8182-43c6-88f2-caefdbe49851 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:35.577453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b5e43d6b-8182-43c6-88f2-caefdbe49851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:35.660919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 447/892] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:35.680686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb9522-4f8a-487b-8629-f8f6e694042d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:35.790685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb9522-4f8a-487b-8629-f8f6e694042d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:35.825617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8478a-c949-44ee-9bb0-1d22acfb9b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:35.826268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8478a-c949-44ee-9bb0-1d22acfb9b82 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:35.826556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8478a-c949-44ee-9bb0-1d22acfb9b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:35.826908 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8478a-c949-44ee-9bb0-1d22acfb9b82 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:35.827564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8478a-c949-44ee-9bb0-1d22acfb9b82 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:35.828966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8478a-c949-44ee-9bb0-1d22acfb9b82 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-14a8478a-c949-44ee-9bb0-1d22acfb9b82 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:35.841060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:35.842164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:35.843676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 446/893] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:35.848131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e43d6b-8182-43c6-88f2-caefdbe49851 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:35.889159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:35.950796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e43d6b-8182-43c6-88f2-caefdbe49851 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:35.986920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffcef0-2130-4daa-ae4c-4df5d6b5f7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:35.987585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffcef0-2130-4daa-ae4c-4df5d6b5f7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:35.987864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffcef0-2130-4daa-ae4c-4df5d6b5f7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:35.988137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffcef0-2130-4daa-ae4c-4df5d6b5f7c2 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:35.988870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffcef0-2130-4daa-ae4c-4df5d6b5f7c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:35.989177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffcef0-2130-4daa-ae4c-4df5d6b5f7c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:35.990535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffcef0-2130-4daa-ae4c-4df5d6b5f7c2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:35.995742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 448/894] 10.222.0.45 () {56 vars in 1122 bytes} [Tue Jul 5 15:51:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:36.002028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8478a-c949-44ee-9bb0-1d22acfb9b82 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:36.039594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:36.108259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8478a-c949-44ee-9bb0-1d22acfb9b82 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:36.143555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc87de-4272-478d-bc90-4787d6d6f59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:36.144225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc87de-4272-478d-bc90-4787d6d6f59e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:36.144510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc87de-4272-478d-bc90-4787d6d6f59e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:36.144806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc87de-4272-478d-bc90-4787d6d6f59e None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:36.145518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc87de-4272-478d-bc90-4787d6d6f59e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:36.145825 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc87de-4272-478d-bc90-4787d6d6f59e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:36.147193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc87de-4272-478d-bc90-4787d6d6f59e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:36.152284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 447/895] 10.222.0.45 () {56 vars in 1122 bytes} [Tue Jul 5 15:51:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:36.498879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184c640-9780-4d51-971b-170efdc6eaa2 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:36.499382 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184c640-9780-4d51-971b-170efdc6eaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:36.499773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184c640-9780-4d51-971b-170efdc6eaa2 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:36.501591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 449/896] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:36.506850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca116ad-1f70-4b27-b6c3-4ce862094b5b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:36.507253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca116ad-1f70-4b27-b6c3-4ce862094b5b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:36.507645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca116ad-1f70-4b27-b6c3-4ce862094b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:36.525312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3ca116ad-1f70-4b27-b6c3-4ce862094b5b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:36.526498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3ca116ad-1f70-4b27-b6c3-4ce862094b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:36.596388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 448/897] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:36.609073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fbefb-14de-4cd9-9997-d19bbdb05d34 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:36.609548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fbefb-14de-4cd9-9997-d19bbdb05d34 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:36.609644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fbefb-14de-4cd9-9997-d19bbdb05d34 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:36.611262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 450/898] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:36.616359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab05a-3867-44b5-be26-b704eb57f4c5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:36.616660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab05a-3867-44b5-be26-b704eb57f4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:36.616974 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab05a-3867-44b5-be26-b704eb57f4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:36.619893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fbefb-14de-4cd9-9997-d19bbdb05d34 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:36.633987 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7c9ab05a-3867-44b5-be26-b704eb57f4c5 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:36.635187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7c9ab05a-3867-44b5-be26-b704eb57f4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:36.660977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:36.717077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 449/899] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:36.732001 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:36.739857 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fbefb-14de-4cd9-9997-d19bbdb05d34 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:36.744902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ab05a-3867-44b5-be26-b704eb57f4c5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:36.775405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786deb-5207-478c-a868-361986bf4298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:36.776145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786deb-5207-478c-a868-361986bf4298 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:36.776435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786deb-5207-478c-a868-361986bf4298 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:36.776718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786deb-5207-478c-a868-361986bf4298 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:36.788524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db786deb-5207-478c-a868-361986bf4298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-db786deb-5207-478c-a868-361986bf4298 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:36.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:36.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:36.799719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 451/900] 10.222.0.45 () {56 vars in 1120 bytes} [Tue Jul 5 15:51:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:36.806173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786deb-5207-478c-a868-361986bf4298 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:36.856510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ab05a-3867-44b5-be26-b704eb57f4c5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:36.894863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132a43c-9d54-4aee-aee8-4896d51ff99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:36.895878 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132a43c-9d54-4aee-aee8-4896d51ff99f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:36.896264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132a43c-9d54-4aee-aee8-4896d51ff99f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:36.896637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132a43c-9d54-4aee-aee8-4896d51ff99f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:36.897423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132a43c-9d54-4aee-aee8-4896d51ff99f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:36.899011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132a43c-9d54-4aee-aee8-4896d51ff99f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-5132a43c-9d54-4aee-aee8-4896d51ff99f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:36.912248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:36.914084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:51:36.915661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:36.922226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 450/901] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:51:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:36.923317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132a43c-9d54-4aee-aee8-4896d51ff99f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:36.939895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db786deb-5207-478c-a868-361986bf4298 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:36.955772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:36.986679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b63cd-e4c0-49d2-95ca-8b7d1c94d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:36.988084 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b63cd-e4c0-49d2-95ca-8b7d1c94d502 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:36.988677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b63cd-e4c0-49d2-95ca-8b7d1c94d502 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:36.989260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b63cd-e4c0-49d2-95ca-8b7d1c94d502 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:36.990353 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b63cd-e4c0-49d2-95ca-8b7d1c94d502 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:36.990827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b63cd-e4c0-49d2-95ca-8b7d1c94d502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:36.992928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b63cd-e4c0-49d2-95ca-8b7d1c94d502 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:37.005954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 452/902] 10.222.0.45 () {56 vars in 1115 bytes} [Tue Jul 5 15:51:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:37.011353 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b63cd-e4c0-49d2-95ca-8b7d1c94d502 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:37.042157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132a43c-9d54-4aee-aee8-4896d51ff99f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:37.049149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:37.088607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ba406-9c40-4371-bccc-143509d7bb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:37.089713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ba406-9c40-4371-bccc-143509d7bb68 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:37.090207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ba406-9c40-4371-bccc-143509d7bb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:37.090646 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ba406-9c40-4371-bccc-143509d7bb68 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:37.092007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ba406-9c40-4371-bccc-143509d7bb68 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:37.092490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ba406-9c40-4371-bccc-143509d7bb68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:37.094337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ba406-9c40-4371-bccc-143509d7bb68 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:37.101206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 451/903] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:51:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:37.107946 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ba406-9c40-4371-bccc-143509d7bb68 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:37.138851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b63cd-e4c0-49d2-95ca-8b7d1c94d502 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:37.185883 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900930-842b-44d0-b1f3-6cb9e522fc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:37.186677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900930-842b-44d0-b1f3-6cb9e522fc23 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:37.186965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900930-842b-44d0-b1f3-6cb9e522fc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:37.187243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e900930-842b-44d0-b1f3-6cb9e522fc23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:37.202466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e900930-842b-44d0-b1f3-6cb9e522fc23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:37.204883 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e900930-842b-44d0-b1f3-6cb9e522fc23 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:37.209585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:37.224290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ba406-9c40-4371-bccc-143509d7bb68 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-6e900930-842b-44d0-b1f3-6cb9e522fc23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:37.234079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:37.234951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:37.236493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:37.236493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:37.236493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:37.236493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:37.236493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 453/904] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:37.236493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900930-842b-44d0-b1f3-6cb9e522fc23 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:37.268054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f650b-ad70-454e-9ce8-7c7defea0e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:37.269157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f650b-ad70-454e-9ce8-7c7defea0e55 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:37.269684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f650b-ad70-454e-9ce8-7c7defea0e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:37.269970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f650b-ad70-454e-9ce8-7c7defea0e55 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:37.270737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f650b-ad70-454e-9ce8-7c7defea0e55 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:37.271054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f650b-ad70-454e-9ce8-7c7defea0e55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:37.272986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f650b-ad70-454e-9ce8-7c7defea0e55 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:37.286592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 452/905] 10.222.0.45 () {56 vars in 1114 bytes} [Tue Jul 5 15:51:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:37.290365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f650b-ad70-454e-9ce8-7c7defea0e55 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:37.328263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:37.359628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e900930-842b-44d0-b1f3-6cb9e522fc23 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:37.395613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5eab01-369f-4bb7-9c13-04bc621e4792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:37.396261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eab01-369f-4bb7-9c13-04bc621e4792 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:37.396541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eab01-369f-4bb7-9c13-04bc621e4792 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:37.396841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eab01-369f-4bb7-9c13-04bc621e4792 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:37.397654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5eab01-369f-4bb7-9c13-04bc621e4792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:37.397862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5eab01-369f-4bb7-9c13-04bc621e4792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:37.399213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5eab01-369f-4bb7-9c13-04bc621e4792 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:37.404430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f650b-ad70-454e-9ce8-7c7defea0e55 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:37.416169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 454/906] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:37.416169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407534f-b80b-4063-8426-36f688e0e28b None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:37.416169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407534f-b80b-4063-8426-36f688e0e28b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:37.416169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407534f-b80b-4063-8426-36f688e0e28b None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:37.416169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 455/907] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:37.417179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407534f-b80b-4063-8426-36f688e0e28b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:37.446572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c914583-b79b-48b3-988d-772ec0895b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:37.447227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914583-b79b-48b3-988d-772ec0895b41 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:37.447507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914583-b79b-48b3-988d-772ec0895b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:37.447780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914583-b79b-48b3-988d-772ec0895b41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:37.448644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c914583-b79b-48b3-988d-772ec0895b41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:37.450098 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c914583-b79b-48b3-988d-772ec0895b41 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:37.521307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407534f-b80b-4063-8426-36f688e0e28b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:37.533997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 453/908] 10.222.0.45 () {60 vars in 1144 bytes} [Tue Jul 5 15:51:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:37.546592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5151b-aefd-4fa2-a336-87514b3f0525 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:37.546905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5151b-aefd-4fa2-a336-87514b3f0525 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:37.547287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5151b-aefd-4fa2-a336-87514b3f0525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:37.555929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:37.578612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db6efb-3d03-4b8d-86e2-7307270d20e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:37.579482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db6efb-3d03-4b8d-86e2-7307270d20e6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:37.579736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db6efb-3d03-4b8d-86e2-7307270d20e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:37.580008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db6efb-3d03-4b8d-86e2-7307270d20e6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:37.581176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d1e5151b-aefd-4fa2-a336-87514b3f0525 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:37.582858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d1e5151b-aefd-4fa2-a336-87514b3f0525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:37.586051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:37.588603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db6efb-3d03-4b8d-86e2-7307270d20e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:37.590202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db6efb-3d03-4b8d-86e2-7307270d20e6 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-49db6efb-3d03-4b8d-86e2-7307270d20e6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:37.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:37.609915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:37.611793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:37.611793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:37.611793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:37.611793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:37.611793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:37.611793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:37.611793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:37.611793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:37.611793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:37.611793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:37.615243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 456/909] 10.222.0.45 () {56 vars in 1117 bytes} [Tue Jul 5 15:51:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:37.616383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db6efb-3d03-4b8d-86e2-7307270d20e6 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:37.688932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 454/910] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:37.710557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5151b-aefd-4fa2-a336-87514b3f0525 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:37.759274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db6efb-3d03-4b8d-86e2-7307270d20e6 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:37.804253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36e9cd-1aa8-45b5-a640-81ecd88461ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:37.804253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36e9cd-1aa8-45b5-a640-81ecd88461ec None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:37.804253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36e9cd-1aa8-45b5-a640-81ecd88461ec None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:37.804253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36e9cd-1aa8-45b5-a640-81ecd88461ec None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:37.805473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36e9cd-1aa8-45b5-a640-81ecd88461ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:37.805473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36e9cd-1aa8-45b5-a640-81ecd88461ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:37.807007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36e9cd-1aa8-45b5-a640-81ecd88461ec None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:37.830513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 457/911] 10.222.0.45 () {56 vars in 1149 bytes} [Tue Jul 5 15:51:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:37.836425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36e9cd-1aa8-45b5-a640-81ecd88461ec None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:37.838645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5151b-aefd-4fa2-a336-87514b3f0525 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:37.878100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b692f68-684b-44e0-a149-94fd4a356481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:37.878915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b692f68-684b-44e0-a149-94fd4a356481 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:37.879255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b692f68-684b-44e0-a149-94fd4a356481 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:37.879535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b692f68-684b-44e0-a149-94fd4a356481 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:37.880331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b692f68-684b-44e0-a149-94fd4a356481 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:37.881856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b692f68-684b-44e0-a149-94fd4a356481 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6b692f68-684b-44e0-a149-94fd4a356481 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:37.896114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:37.897248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:37.898083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 455/912] 10.222.0.45 () {56 vars in 1132 bytes} [Tue Jul 5 15:51:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:37.903218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b692f68-684b-44e0-a149-94fd4a356481 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:37.905251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:37.906199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:37.971743 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36e9cd-1aa8-45b5-a640-81ecd88461ec None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:38.011417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2183bd-d15b-4f9e-9e9e-a39b0538f86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:38.012142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2183bd-d15b-4f9e-9e9e-a39b0538f86f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:38.012426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2183bd-d15b-4f9e-9e9e-a39b0538f86f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:38.012706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2183bd-d15b-4f9e-9e9e-a39b0538f86f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:38.032388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b692f68-684b-44e0-a149-94fd4a356481 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:38.070936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b663f-d8d4-42fd-af3b-37bbd9821a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:38.072164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b663f-d8d4-42fd-af3b-37bbd9821a53 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:38.072913 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b663f-d8d4-42fd-af3b-37bbd9821a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:38.073744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b663f-d8d4-42fd-af3b-37bbd9821a53 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:38.074654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b663f-d8d4-42fd-af3b-37bbd9821a53 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:38.075175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b663f-d8d4-42fd-af3b-37bbd9821a53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:38.077309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b663f-d8d4-42fd-af3b-37bbd9821a53 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:38.084835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 456/913] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:38.090560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b663f-d8d4-42fd-af3b-37bbd9821a53 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:38.099835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2183bd-d15b-4f9e-9e9e-a39b0538f86f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:38.101319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2183bd-d15b-4f9e-9e9e-a39b0538f86f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:38.160088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:38.165961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:38.205758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b663f-d8d4-42fd-af3b-37bbd9821a53 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:38.226385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 458/914] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:37 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:51:38.228283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca9980-bf8b-4054-9964-6cfb49a4991c None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:38.228806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca9980-bf8b-4054-9964-6cfb49a4991c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:38.229286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca9980-bf8b-4054-9964-6cfb49a4991c None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:38.234524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 459/915] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:38.234524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca9980-bf8b-4054-9964-6cfb49a4991c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:38.246252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf0cd9-0832-44f5-88bb-d53d8a80d6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:38.246949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf0cd9-0832-44f5-88bb-d53d8a80d6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:38.247236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf0cd9-0832-44f5-88bb-d53d8a80d6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:38.247620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf0cd9-0832-44f5-88bb-d53d8a80d6c4 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:38.248209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf0cd9-0832-44f5-88bb-d53d8a80d6c4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:38.248515 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf0cd9-0832-44f5-88bb-d53d8a80d6c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:38.249904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf0cd9-0832-44f5-88bb-d53d8a80d6c4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:38.259692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 457/916] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:38.259692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5c0d5-a401-4fc3-b7a7-e882e581362d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:38.259692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5c0d5-a401-4fc3-b7a7-e882e581362d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:38.259692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5c0d5-a401-4fc3-b7a7-e882e581362d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:38.275670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-17d5c0d5-a401-4fc3-b7a7-e882e581362d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:38.277254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-17d5c0d5-a401-4fc3-b7a7-e882e581362d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:38.347963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 458/917] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:38.349893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5c0d5-a401-4fc3-b7a7-e882e581362d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:38.354258 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca9980-bf8b-4054-9964-6cfb49a4991c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:38.392409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cd91f-6286-45e0-bcac-f604b9a358a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:38.393244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cd91f-6286-45e0-bcac-f604b9a358a8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:38.393528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cd91f-6286-45e0-bcac-f604b9a358a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:38.393804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cd91f-6286-45e0-bcac-f604b9a358a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:38.404285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cd91f-6286-45e0-bcac-f604b9a358a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:38.405724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cd91f-6286-45e0-bcac-f604b9a358a8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:38.408982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-ce3cd91f-6286-45e0-bcac-f604b9a358a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:38.424210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:38.430998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:38.432080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:38.432080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:38.432080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:38.432080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:38.432080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 460/918] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:38.432080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cd91f-6286-45e0-bcac-f604b9a358a8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:38.433580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:38.476496 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5c0d5-a401-4fc3-b7a7-e882e581362d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:38.512282 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95903842-c78a-476f-afd0-575a1c4a2165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:38.512961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95903842-c78a-476f-afd0-575a1c4a2165 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:38.513237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95903842-c78a-476f-afd0-575a1c4a2165 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:38.513507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95903842-c78a-476f-afd0-575a1c4a2165 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:38.514235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95903842-c78a-476f-afd0-575a1c4a2165 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:38.515786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95903842-c78a-476f-afd0-575a1c4a2165 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:38.534618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cd91f-6286-45e0-bcac-f604b9a358a8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:38.568259 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d43cc5-d33c-45dc-9446-44a02253fbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:38.568966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43cc5-d33c-45dc-9446-44a02253fbd4 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:38.569250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43cc5-d33c-45dc-9446-44a02253fbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:38.569528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43cc5-d33c-45dc-9446-44a02253fbd4 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:38.570248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d43cc5-d33c-45dc-9446-44a02253fbd4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:38.571639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d43cc5-d33c-45dc-9446-44a02253fbd4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:38.606758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 459/919] 10.222.0.45 () {60 vars in 1144 bytes} [Tue Jul 5 15:51:38 2022] POST /identity/v3/endpoints => generated 347 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:38.607507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95903842-c78a-476f-afd0-575a1c4a2165 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:38.665200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 461/920] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:51:38 2022] POST /identity/v3/services => generated 241 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:38.741710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95903842-c78a-476f-afd0-575a1c4a2165 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:38.758942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:38.791741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccc545-418e-4196-8916-e6807fdd8b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:38.792433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccc545-418e-4196-8916-e6807fdd8b90 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:38.792715 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccc545-418e-4196-8916-e6807fdd8b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:38.793020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccc545-418e-4196-8916-e6807fdd8b90 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:38.793770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccc545-418e-4196-8916-e6807fdd8b90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:38.794079 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccc545-418e-4196-8916-e6807fdd8b90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:38.795496 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccc545-418e-4196-8916-e6807fdd8b90 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:38.809476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 460/921] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:38.815721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d43cc5-d33c-45dc-9446-44a02253fbd4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:38.878871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:38.926469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d43cc5-d33c-45dc-9446-44a02253fbd4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:38.966751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc00541-5fa3-435d-b99d-e02cd0fe8c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:38.967549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc00541-5fa3-435d-b99d-e02cd0fe8c78 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:38.967826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc00541-5fa3-435d-b99d-e02cd0fe8c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:38.968099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc00541-5fa3-435d-b99d-e02cd0fe8c78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:38.974962 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc00541-5fa3-435d-b99d-e02cd0fe8c78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:38.976384 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc00541-5fa3-435d-b99d-e02cd0fe8c78 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-9cc00541-5fa3-435d-b99d-e02cd0fe8c78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:38.991433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:38.992334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:38.993230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:38.993230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:38.993230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:38.993230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:38.993230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:38.993230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:38.993230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:38.993230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:38.993230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 462/922] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:38.999917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccc545-418e-4196-8916-e6807fdd8b90 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:39.105457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccc545-418e-4196-8916-e6807fdd8b90 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:39.141404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2fe6-3136-475f-a718-b74a7b9195c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:39.142439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2fe6-3136-475f-a718-b74a7b9195c5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:39.142925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2fe6-3136-475f-a718-b74a7b9195c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:39.143426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2fe6-3136-475f-a718-b74a7b9195c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:39.144436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca2fe6-3136-475f-a718-b74a7b9195c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:39.145004 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca2fe6-3136-475f-a718-b74a7b9195c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:39.146641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca2fe6-3136-475f-a718-b74a7b9195c5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:39.160207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 461/923] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:39.166481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc00541-5fa3-435d-b99d-e02cd0fe8c78 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:39.274312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc00541-5fa3-435d-b99d-e02cd0fe8c78 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:39.296791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:39.315600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d36ffc-691b-423e-baeb-00775d6a32a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:39.316249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d36ffc-691b-423e-baeb-00775d6a32a3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:39.316531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d36ffc-691b-423e-baeb-00775d6a32a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:39.316827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d36ffc-691b-423e-baeb-00775d6a32a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:39.317661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d36ffc-691b-423e-baeb-00775d6a32a3 None admin] RBAC: Adding query filter params (role.id=450b100797c74a58b92e58120d0c8387, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=7245effe463b45dfa946ca2eda2e0c98) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:39.317969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d36ffc-691b-423e-baeb-00775d6a32a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:39.319388 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d36ffc-691b-423e-baeb-00775d6a32a3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:39.325702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 463/924] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:39 2022] GET /identity/v3/role_assignments?user.id=7245effe463b45dfa946ca2eda2e0c98&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=450b100797c74a58b92e58120d0c8387 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:39.728128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996e29e-5429-49fb-b3be-506a81f77235 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:39.729002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996e29e-5429-49fb-b3be-506a81f77235 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:39.729518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996e29e-5429-49fb-b3be-506a81f77235 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:39.731319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 462/925] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:39.736553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656477-c616-46ca-8bb6-5dfd65f0230b None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:39.737422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656477-c616-46ca-8bb6-5dfd65f0230b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:39.737910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656477-c616-46ca-8bb6-5dfd65f0230b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:39.754832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-aa656477-c616-46ca-8bb6-5dfd65f0230b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:39.756294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-aa656477-c616-46ca-8bb6-5dfd65f0230b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:39.826609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 464/926] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:39.851078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996e29e-5429-49fb-b3be-506a81f77235 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:39.881351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:39.965127 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996e29e-5429-49fb-b3be-506a81f77235 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:40.002892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:40.003503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:40.003745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:40.004019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:40.004730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:40.006249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:40.007618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WARNING keystone.server.flask.common [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 05 15:51:40.052029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.sql.core [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 05 15:51:40.052029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 05 15:51:40.052029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [parameters: {'id': '7eda58fa7f114b67b1bb5d8eeb11a20d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 05 15:51:40.052029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80398) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7eda58fa7f114b67b1bb5d8eeb11a20d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7eda58fa7f114b67b1bb5d8eeb11a20d. Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:51:40.059812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result.read() Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 05 15:51:40.060597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application yield resource Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.session.rollback() Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.force_reraise() Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise self.value Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application session.commit() Jul 05 15:51:40.061566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._prepare_impl() Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.session.flush() Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._flush(objects) Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application compat.raise_( Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application flush_context.execute() Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 05 15:51:40.062433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rec.execute(self) Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 05 15:51:40.063313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result.read() Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:51:40.064624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [parameters: {'id': '7eda58fa7f114b67b1bb5d8eeb11a20d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:40.067359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:40.073450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:40.075565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:40.075565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 05 15:51:40.075565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 05 15:51:40.075565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:40.075565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:40.075565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 05 15:51:40.075565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 05 15:51:40.075565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7eda58fa7f114b67b1bb5d8eeb11a20d. Jul 05 15:51:40.075565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:40.075565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 463/927] 10.222.0.45 () {60 vars in 1140 bytes} [Tue Jul 5 15:51:39 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:40.081117 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa656477-c616-46ca-8bb6-5dfd65f0230b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:40.181785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa656477-c616-46ca-8bb6-5dfd65f0230b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:40.194053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:40.217832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b22f5-1e7f-42d0-9775-b2619e85e902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:40.218614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b22f5-1e7f-42d0-9775-b2619e85e902 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:40.218892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b22f5-1e7f-42d0-9775-b2619e85e902 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:40.219175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b22f5-1e7f-42d0-9775-b2619e85e902 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:40.226119 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b22f5-1e7f-42d0-9775-b2619e85e902 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:40.227809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b22f5-1e7f-42d0-9775-b2619e85e902 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-919b22f5-1e7f-42d0-9775-b2619e85e902 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:40.241963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:40.242995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:40.243907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:40.243907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:40.243907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:40.243907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:40.243907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:40.243907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:40.243907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:40.243907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:40.243907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 465/928] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:40.249055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:40.356125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:40.372868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fc440-47d5-41f0-bc88-217847d54ab7 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:40.408158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc37700-ac1d-4343-964d-bf099b7e13f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:40.408846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc37700-ac1d-4343-964d-bf099b7e13f6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:40.409133 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc37700-ac1d-4343-964d-bf099b7e13f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:40.409413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc37700-ac1d-4343-964d-bf099b7e13f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:40.410181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc37700-ac1d-4343-964d-bf099b7e13f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:40.410542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc37700-ac1d-4343-964d-bf099b7e13f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:40.412198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc37700-ac1d-4343-964d-bf099b7e13f6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:40.433165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 464/929] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:40.769717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c434cc-030d-4d84-b79b-c607e8b82182 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:40.770235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c434cc-030d-4d84-b79b-c607e8b82182 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:40.770312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c434cc-030d-4d84-b79b-c607e8b82182 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:40.771842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 466/930] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:40.777217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d326e4a-73dd-42c0-b6ce-c4a8697dddc4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:40.777780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d326e4a-73dd-42c0-b6ce-c4a8697dddc4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:40.778276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d326e4a-73dd-42c0-b6ce-c4a8697dddc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:40.796181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1d326e4a-73dd-42c0-b6ce-c4a8697dddc4 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:40.798621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1d326e4a-73dd-42c0-b6ce-c4a8697dddc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:40.868005 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 465/931] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:40.890719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c434cc-030d-4d84-b79b-c607e8b82182 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:40.911312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ad33d-5ea1-4c90-a7ed-d8cfe002e569 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:40.911621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ad33d-5ea1-4c90-a7ed-d8cfe002e569 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:40.911879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ad33d-5ea1-4c90-a7ed-d8cfe002e569 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:40.913884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 466/932] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:40.920248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4dbba-0b15-4def-b666-405b38288c39 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:40.920562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4dbba-0b15-4def-b666-405b38288c39 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:40.921048 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4dbba-0b15-4def-b666-405b38288c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:40.943560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9ce4dbba-0b15-4def-b666-405b38288c39 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:40.946417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9ce4dbba-0b15-4def-b666-405b38288c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:41.009769 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c434cc-030d-4d84-b79b-c607e8b82182 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:41.036893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 467/933] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:41.057403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f0340-2af2-4cf6-b967-9738b227fe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:41.058538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f0340-2af2-4cf6-b967-9738b227fe44 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.059025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f0340-2af2-4cf6-b967-9738b227fe44 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.060465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4dbba-0b15-4def-b666-405b38288c39 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:41.060986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f0340-2af2-4cf6-b967-9738b227fe44 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.061891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f0340-2af2-4cf6-b967-9738b227fe44 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:41.064106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f0340-2af2-4cf6-b967-9738b227fe44 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:41.064216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-e80f0340-2af2-4cf6-b967-9738b227fe44 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:41.077226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:41.084613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:41.085473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 467/934] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:41.088234 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f0340-2af2-4cf6-b967-9738b227fe44 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:41.125870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:41.188071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4dbba-0b15-4def-b666-405b38288c39 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:41.208662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f0340-2af2-4cf6-b967-9738b227fe44 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:41.225717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbce66-a70c-4f88-bde8-21c77e399f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:41.227197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbce66-a70c-4f88-bde8-21c77e399f36 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.227679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbce66-a70c-4f88-bde8-21c77e399f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.228164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbce66-a70c-4f88-bde8-21c77e399f36 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.229074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbce66-a70c-4f88-bde8-21c77e399f36 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:41.230640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefbce66-a70c-4f88-bde8-21c77e399f36 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-fefbce66-a70c-4f88-bde8-21c77e399f36 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:41.248472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:41.249674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:41.250498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:41.253057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bbddc-e8be-4dbf-9038-3a73d4947c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:41.253943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bbddc-e8be-4dbf-9038-3a73d4947c43 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.254420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bbddc-e8be-4dbf-9038-3a73d4947c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.255337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 468/935] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:41.255561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bbddc-e8be-4dbf-9038-3a73d4947c43 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.256223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbce66-a70c-4f88-bde8-21c77e399f36 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:41.256853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bbddc-e8be-4dbf-9038-3a73d4947c43 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:41.257372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bbddc-e8be-4dbf-9038-3a73d4947c43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:41.258943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bbddc-e8be-4dbf-9038-3a73d4947c43 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:41.268938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 468/936] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:41.275845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bbddc-e8be-4dbf-9038-3a73d4947c43 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:41.342927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:41.382528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbce66-a70c-4f88-bde8-21c77e399f36 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:41.416726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:41.418867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ee4d7-7f73-4c1b-bd50-5e25105fd371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:41.419538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ee4d7-7f73-4c1b-bd50-5e25105fd371 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.419824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ee4d7-7f73-4c1b-bd50-5e25105fd371 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.420100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ee4d7-7f73-4c1b-bd50-5e25105fd371 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.420196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bbddc-e8be-4dbf-9038-3a73d4947c43 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:41.420854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ee4d7-7f73-4c1b-bd50-5e25105fd371 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:41.421167 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ee4d7-7f73-4c1b-bd50-5e25105fd371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:41.423445 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ee4d7-7f73-4c1b-bd50-5e25105fd371 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:41.428319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 469/937] 10.222.0.45 () {56 vars in 1122 bytes} [Tue Jul 5 15:51:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:41.431786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb1b1d-3c84-4193-a81a-51ea0be5030e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.432074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb1b1d-3c84-4193-a81a-51ea0be5030e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.432339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb1b1d-3c84-4193-a81a-51ea0be5030e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.434920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 470/938] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:41.435601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb1b1d-3c84-4193-a81a-51ea0be5030e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:41.461507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842de481-2535-46ec-bcc5-afc6fd1f351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:41.462320 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842de481-2535-46ec-bcc5-afc6fd1f351c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.462937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842de481-2535-46ec-bcc5-afc6fd1f351c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.463229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842de481-2535-46ec-bcc5-afc6fd1f351c None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.464095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842de481-2535-46ec-bcc5-afc6fd1f351c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:41.464418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842de481-2535-46ec-bcc5-afc6fd1f351c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:41.465809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842de481-2535-46ec-bcc5-afc6fd1f351c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:41.480813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 469/939] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:41.480813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a86a0-fa3b-4fa9-832f-8c866f602d48 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.480813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a86a0-fa3b-4fa9-832f-8c866f602d48 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.480813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a86a0-fa3b-4fa9-832f-8c866f602d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.498614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f00a86a0-fa3b-4fa9-832f-8c866f602d48 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:41.500135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f00a86a0-fa3b-4fa9-832f-8c866f602d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:41.570616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb1b1d-3c84-4193-a81a-51ea0be5030e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:41.588059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 470/940] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:41.611778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a86a0-fa3b-4fa9-832f-8c866f602d48 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:41.617571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:41.620621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee81d52-615e-4721-b0ba-08ae17d2a00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:41.621612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee81d52-615e-4721-b0ba-08ae17d2a00d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.622053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee81d52-615e-4721-b0ba-08ae17d2a00d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.622498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee81d52-615e-4721-b0ba-08ae17d2a00d None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.623402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee81d52-615e-4721-b0ba-08ae17d2a00d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:41.623912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee81d52-615e-4721-b0ba-08ae17d2a00d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:41.625669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee81d52-615e-4721-b0ba-08ae17d2a00d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:41.636968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 471/941] 10.222.0.45 () {56 vars in 1122 bytes} [Tue Jul 5 15:51:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:41.638613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee81d52-615e-4721-b0ba-08ae17d2a00d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:41.722248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:41.732030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a86a0-fa3b-4fa9-832f-8c866f602d48 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:41.774164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee81d52-615e-4721-b0ba-08ae17d2a00d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:41.775234 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2df211-27e1-4ef9-837e-63e4ab9e8858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:41.776686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2df211-27e1-4ef9-837e-63e4ab9e8858 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.777246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2df211-27e1-4ef9-837e-63e4ab9e8858 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.777744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2df211-27e1-4ef9-837e-63e4ab9e8858 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.789670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2df211-27e1-4ef9-837e-63e4ab9e8858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-3d2df211-27e1-4ef9-837e-63e4ab9e8858 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:41.798601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:41.799538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:51:41.800377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:41.807877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 471/942] 10.222.0.45 () {56 vars in 1120 bytes} [Tue Jul 5 15:51:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:41.809148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2df211-27e1-4ef9-837e-63e4ab9e8858 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:41.810084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8de81-d617-42a0-8d84-e158e0db653a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:41.810956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8de81-d617-42a0-8d84-e158e0db653a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.811247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8de81-d617-42a0-8d84-e158e0db653a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.811524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8de81-d617-42a0-8d84-e158e0db653a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.812316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8de81-d617-42a0-8d84-e158e0db653a None admin] RBAC: Adding query filter params (interface=public, service_id=7512bb97a93a41d5a7af17bbddb1ebd5, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:41.812719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8de81-d617-42a0-8d84-e158e0db653a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:41.815118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8de81-d617-42a0-8d84-e158e0db653a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:41.837207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 472/943] 10.222.0.45 () {56 vars in 1251 bytes} [Tue Jul 5 15:51:41 2022] GET /identity/v3/endpoints?service_id=7512bb97a93a41d5a7af17bbddb1ebd5&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:41.837207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8de81-d617-42a0-8d84-e158e0db653a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:41.892818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:41.928359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2df211-27e1-4ef9-837e-63e4ab9e8858 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:41.937587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8de81-d617-42a0-8d84-e158e0db653a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:41.965438 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e147e-8db7-4a10-a5ec-570a2d0bdfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:41.966487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e147e-8db7-4a10-a5ec-570a2d0bdfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.966999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e147e-8db7-4a10-a5ec-570a2d0bdfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.967477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e147e-8db7-4a10-a5ec-570a2d0bdfc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.968430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e147e-8db7-4a10-a5ec-570a2d0bdfc4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:41.968992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e147e-8db7-4a10-a5ec-570a2d0bdfc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:41.971146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e147e-8db7-4a10-a5ec-570a2d0bdfc4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:41.975977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e55fc-210c-4334-bcc1-18980b162a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:41.976951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e55fc-210c-4334-bcc1-18980b162a8b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:41.977234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e55fc-210c-4334-bcc1-18980b162a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:41.977509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e55fc-210c-4334-bcc1-18980b162a8b None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:41.978193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e55fc-210c-4334-bcc1-18980b162a8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:41.979555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e55fc-210c-4334-bcc1-18980b162a8b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:41.988789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 473/944] 10.222.0.45 () {56 vars in 1087 bytes} [Tue Jul 5 15:51:41 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:51:41.988958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 472/945] 10.222.0.45 () {56 vars in 1115 bytes} [Tue Jul 5 15:51:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:42.003151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e55fc-210c-4334-bcc1-18980b162a8b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:42.117849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e55fc-210c-4334-bcc1-18980b162a8b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:42.144220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:42.157439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f0bfa-a546-4ba1-8d78-8a971d16f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:42.158254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f0bfa-a546-4ba1-8d78-8a971d16f245 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:42.158799 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f0bfa-a546-4ba1-8d78-8a971d16f245 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:42.159076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f0bfa-a546-4ba1-8d78-8a971d16f245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:42.170274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f0bfa-a546-4ba1-8d78-8a971d16f245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:42.171725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f0bfa-a546-4ba1-8d78-8a971d16f245 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-183f0bfa-a546-4ba1-8d78-8a971d16f245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:42.190233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:42.191336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:42.192438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:42.192438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:42.192438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:42.192438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:42.194719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 474/946] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:42.201832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e147e-8db7-4a10-a5ec-570a2d0bdfc4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:42.219955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:42.318809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e147e-8db7-4a10-a5ec-570a2d0bdfc4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:42.353655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee60f7-03df-4f01-a00d-72be969aafb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:42.354312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee60f7-03df-4f01-a00d-72be969aafb5 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:42.354592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee60f7-03df-4f01-a00d-72be969aafb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:42.354865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee60f7-03df-4f01-a00d-72be969aafb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:42.355583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee60f7-03df-4f01-a00d-72be969aafb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:42.355884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee60f7-03df-4f01-a00d-72be969aafb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:42.357274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee60f7-03df-4f01-a00d-72be969aafb5 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:42.369438 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 473/947] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:42.374887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f0bfa-a546-4ba1-8d78-8a971d16f245 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:42.480901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f0bfa-a546-4ba1-8d78-8a971d16f245 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:42.516537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c8478-13bb-477e-9fe0-875b88c2dd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:42.517379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c8478-13bb-477e-9fe0-875b88c2dd60 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:42.517663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c8478-13bb-477e-9fe0-875b88c2dd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:42.517937 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c8478-13bb-477e-9fe0-875b88c2dd60 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:42.525828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c8478-13bb-477e-9fe0-875b88c2dd60 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:42.527279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c8478-13bb-477e-9fe0-875b88c2dd60 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-709c8478-13bb-477e-9fe0-875b88c2dd60 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:42.541965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:42.542900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:42.543835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 475/948] 10.222.0.45 () {56 vars in 1117 bytes} [Tue Jul 5 15:51:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:42.551087 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee60f7-03df-4f01-a00d-72be969aafb5 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:42.570287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668b8f-9cfb-4db5-9284-e526f06e6921 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:42.570594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668b8f-9cfb-4db5-9284-e526f06e6921 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:42.570851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668b8f-9cfb-4db5-9284-e526f06e6921 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:42.572840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 476/949] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:51:42.577338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e251d37-358d-4096-a3ef-1db6f704f9ad None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:42.577653 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e251d37-358d-4096-a3ef-1db6f704f9ad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:42.577939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e251d37-358d-4096-a3ef-1db6f704f9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:42.596470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8e251d37-358d-4096-a3ef-1db6f704f9ad None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:42.597715 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8e251d37-358d-4096-a3ef-1db6f704f9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:42.630865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:42.645440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:42.671505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee60f7-03df-4f01-a00d-72be969aafb5 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:42.678868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 477/950] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:42.703663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e251d37-358d-4096-a3ef-1db6f704f9ad None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:42.715923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0f0d1-ea92-4a02-9e82-3233ebafaab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:42.717133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0f0d1-ea92-4a02-9e82-3233ebafaab6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:42.717603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0f0d1-ea92-4a02-9e82-3233ebafaab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:42.718224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0f0d1-ea92-4a02-9e82-3233ebafaab6 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:42.719731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0f0d1-ea92-4a02-9e82-3233ebafaab6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:42.720520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0f0d1-ea92-4a02-9e82-3233ebafaab6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:42.723009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0f0d1-ea92-4a02-9e82-3233ebafaab6 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:42.748211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 474/951] 10.222.0.45 () {56 vars in 1149 bytes} [Tue Jul 5 15:51:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:42.754722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0f0d1-ea92-4a02-9e82-3233ebafaab6 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:42.829468 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e251d37-358d-4096-a3ef-1db6f704f9ad None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:42.864116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0f0d1-ea92-4a02-9e82-3233ebafaab6 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:42.873669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32366097-63b3-45d7-910f-94174cd210b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:42.874879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32366097-63b3-45d7-910f-94174cd210b0 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:42.875377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32366097-63b3-45d7-910f-94174cd210b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:42.876130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32366097-63b3-45d7-910f-94174cd210b0 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:42.877361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32366097-63b3-45d7-910f-94174cd210b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:42.879144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32366097-63b3-45d7-910f-94174cd210b0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:42.896539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:42.901672 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c65f9-590f-49ba-98ac-6c925fcbfd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:42.904378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c65f9-590f-49ba-98ac-6c925fcbfd68 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:42.904378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c65f9-590f-49ba-98ac-6c925fcbfd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:42.904378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c65f9-590f-49ba-98ac-6c925fcbfd68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:42.913812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c65f9-590f-49ba-98ac-6c925fcbfd68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:42.915313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c65f9-590f-49ba-98ac-6c925fcbfd68 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:42.929243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.sql.core [None req-32366097-63b3-45d7-910f-94174cd210b0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 05 15:51:42.929243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 05 15:51:42.929243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [parameters: {'user_id': '6ef1163395ed4cba81abdb0767f4335d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 05 15:51:42.929243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80398) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-324c65f9-590f-49ba-98ac-6c925fcbfd68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:42.934037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:42.941865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:42.942920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:42.942920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:42.942920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:42.942920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:42.942920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 475/952] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:42.942920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c65f9-590f-49ba-98ac-6c925fcbfd68 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-32366097-63b3-45d7-910f-94174cd210b0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result.read() Jul 05 15:51:42.943757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:51:42.944673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application yield resource Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.session.rollback() Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.force_reraise() Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise self.value Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application session.commit() Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 05 15:51:42.946545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._prepare_impl() Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.session.flush() Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._flush(objects) Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application compat.raise_( Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application flush_context.execute() Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rec.execute(self) Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 05 15:51:42.949902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:51:42.952244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application result.read() Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:51:42.954506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [parameters: {'user_id': '6ef1163395ed4cba81abdb0767f4335d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:42.957648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 05 15:51:42.959647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 05 15:51:42.965071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 05 15:51:42.965071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 05 15:51:42.965071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 05 15:51:42.965071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 05 15:51:42.965071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 05 15:51:42.965071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:42.965071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 478/953] 10.222.0.45 () {60 vars in 1131 bytes} [Tue Jul 5 15:51:42 2022] POST /identity/v3/users => generated 160 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:42.965071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32366097-63b3-45d7-910f-94174cd210b0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:42.986092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:43.062746 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32366097-63b3-45d7-910f-94174cd210b0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:43.098998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf6672-f59b-4294-9e8a-9f9844cf302f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:43.099595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c65f9-590f-49ba-98ac-6c925fcbfd68 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:43.099816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf6672-f59b-4294-9e8a-9f9844cf302f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:43.100074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf6672-f59b-4294-9e8a-9f9844cf302f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:43.100346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf6672-f59b-4294-9e8a-9f9844cf302f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:43.109002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf6672-f59b-4294-9e8a-9f9844cf302f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:43.111388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf6672-f59b-4294-9e8a-9f9844cf302f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-31bf6672-f59b-4294-9e8a-9f9844cf302f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:43.126990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:43.128394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:43.129403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:43.129403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:43.129403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:43.129403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:43.129403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:43.129403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:43.129403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:43.129403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:43.129403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:51:43.129403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:43.132971 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 479/954] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:43.134235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf6672-f59b-4294-9e8a-9f9844cf302f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:43.147925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0c3e7-c1a5-42c1-a83b-6977d131b7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:43.149178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0c3e7-c1a5-42c1-a83b-6977d131b7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:43.149669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0c3e7-c1a5-42c1-a83b-6977d131b7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:43.150143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0c3e7-c1a5-42c1-a83b-6977d131b7f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:43.151198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0c3e7-c1a5-42c1-a83b-6977d131b7f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:43.152067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0c3e7-c1a5-42c1-a83b-6977d131b7f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:43.153972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0c3e7-c1a5-42c1-a83b-6977d131b7f0 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:43.176802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 476/955] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:43.183371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0c3e7-c1a5-42c1-a83b-6977d131b7f0 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:43.237233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:43.258358 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:43.277907 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf6672-f59b-4294-9e8a-9f9844cf302f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:43.312225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0c3e7-c1a5-42c1-a83b-6977d131b7f0 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:43.327798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24bba7-78a5-4377-87f0-6e22b8afd78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:43.328575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24bba7-78a5-4377-87f0-6e22b8afd78e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:43.329671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24bba7-78a5-4377-87f0-6e22b8afd78e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:43.329960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24bba7-78a5-4377-87f0-6e22b8afd78e None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:43.330778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24bba7-78a5-4377-87f0-6e22b8afd78e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:43.331089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24bba7-78a5-4377-87f0-6e22b8afd78e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:43.333236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24bba7-78a5-4377-87f0-6e22b8afd78e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:43.353078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd4f06-0e34-4626-b6de-a6536517db6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:43.354225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd4f06-0e34-4626-b6de-a6536517db6b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:43.354711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd4f06-0e34-4626-b6de-a6536517db6b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:43.355201 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd4f06-0e34-4626-b6de-a6536517db6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:43.362606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd4f06-0e34-4626-b6de-a6536517db6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:43.364321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd4f06-0e34-4626-b6de-a6536517db6b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-9afd4f06-0e34-4626-b6de-a6536517db6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:43.379886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:43.381970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:43.382893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:43.382893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:43.382893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:43.382893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:43.382893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:43.382893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:43.382893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:43.382893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:43.383338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 480/956] 10.222.0.45 () {56 vars in 1101 bytes} [Tue Jul 5 15:51:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:43.386490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 477/957] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:43.393673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24bba7-78a5-4377-87f0-6e22b8afd78e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:43.510013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24bba7-78a5-4377-87f0-6e22b8afd78e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:43.551657 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851dbf1-3877-481a-b293-a5b8cb655232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:43.552712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851dbf1-3877-481a-b293-a5b8cb655232 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:43.553251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851dbf1-3877-481a-b293-a5b8cb655232 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:43.553756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851dbf1-3877-481a-b293-a5b8cb655232 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:43.554564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851dbf1-3877-481a-b293-a5b8cb655232 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:43.554881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851dbf1-3877-481a-b293-a5b8cb655232 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:43.556330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851dbf1-3877-481a-b293-a5b8cb655232 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:43.569159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 481/958] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:43.574932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd4f06-0e34-4626-b6de-a6536517db6b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:43.591691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1e95e-ff45-499f-af16-1e13648be06c None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:43.592020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1e95e-ff45-499f-af16-1e13648be06c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:43.592315 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1e95e-ff45-499f-af16-1e13648be06c None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:43.593887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 482/959] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:51:43.599711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b521-74a7-40be-81bc-234733c9de4a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:43.600097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b521-74a7-40be-81bc-234733c9de4a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:43.600381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b521-74a7-40be-81bc-234733c9de4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:43.613424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:43.618991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-85d9b521-74a7-40be-81bc-234733c9de4a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:43.620612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-85d9b521-74a7-40be-81bc-234733c9de4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:43.687877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd4f06-0e34-4626-b6de-a6536517db6b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:43.691473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:43.698850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 483/960] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:43.721673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b521-74a7-40be-81bc-234733c9de4a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:43.725007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdcb84-954f-4535-9a44-1fec1290a883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:43.725899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcb84-954f-4535-9a44-1fec1290a883 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:43.726354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcb84-954f-4535-9a44-1fec1290a883 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:43.727278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcb84-954f-4535-9a44-1fec1290a883 None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/7245effe463b45dfa946ca2eda2e0c98/roles/450b100797c74a58b92e58120d0c8387` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:43.730143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdcb84-954f-4535-9a44-1fec1290a883 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=7245effe463b45dfa946ca2eda2e0c98, role_id=450b100797c74a58b92e58120d0c8387)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:43.731644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdcb84-954f-4535-9a44-1fec1290a883 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:43.821094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 478/961] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:51:43 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/7245effe463b45dfa946ca2eda2e0c98/roles/450b100797c74a58b92e58120d0c8387 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:51:43.826220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b521-74a7-40be-81bc-234733c9de4a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:43.866734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e982f-1ad6-473a-ad00-8555081cd58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:43.867417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e982f-1ad6-473a-ad00-8555081cd58b None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:43.867724 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e982f-1ad6-473a-ad00-8555081cd58b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:43.867967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e982f-1ad6-473a-ad00-8555081cd58b None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:43.868815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e982f-1ad6-473a-ad00-8555081cd58b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:43.870235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e982f-1ad6-473a-ad00-8555081cd58b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:43.873848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:43.981978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 484/962] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:51:43 2022] POST /identity/v3/services => generated 242 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:51:44.100440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad99fd-7542-4b91-a2fd-f431e00ed1e6 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:44.100996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad99fd-7542-4b91-a2fd-f431e00ed1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:44.100996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad99fd-7542-4b91-a2fd-f431e00ed1e6 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:44.102758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 479/963] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:44.108149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de1596-e65f-4fbf-9318-228ddfe185ef None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:44.108444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de1596-e65f-4fbf-9318-228ddfe185ef None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:44.108725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de1596-e65f-4fbf-9318-228ddfe185ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:44.127464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e0de1596-e65f-4fbf-9318-228ddfe185ef None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:44.128668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e0de1596-e65f-4fbf-9318-228ddfe185ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:44.205504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 485/964] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:44.227940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad99fd-7542-4b91-a2fd-f431e00ed1e6 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:44.339370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad99fd-7542-4b91-a2fd-f431e00ed1e6 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:44.374325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172a1ec-9b42-40d1-b7e4-47e7166a9521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:44.375366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172a1ec-9b42-40d1-b7e4-47e7166a9521 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:44.375825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172a1ec-9b42-40d1-b7e4-47e7166a9521 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:44.376339 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172a1ec-9b42-40d1-b7e4-47e7166a9521 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:44.377181 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172a1ec-9b42-40d1-b7e4-47e7166a9521 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:44.378682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172a1ec-9b42-40d1-b7e4-47e7166a9521 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-0172a1ec-9b42-40d1-b7e4-47e7166a9521 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:44.390644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:44.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:44.392940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:44.395039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 480/965] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:44.401003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de1596-e65f-4fbf-9318-228ddfe185ef None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:44.500722 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de1596-e65f-4fbf-9318-228ddfe185ef None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:44.524952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:44.537126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0381be6-d381-44c8-a648-b96482541cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:44.537862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0381be6-d381-44c8-a648-b96482541cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:44.538213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0381be6-d381-44c8-a648-b96482541cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:44.538553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0381be6-d381-44c8-a648-b96482541cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:44.539416 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0381be6-d381-44c8-a648-b96482541cb5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:44.539792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0381be6-d381-44c8-a648-b96482541cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:44.541259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0381be6-d381-44c8-a648-b96482541cb5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:44.553323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 486/966] 10.222.0.45 () {56 vars in 1122 bytes} [Tue Jul 5 15:51:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:44.553545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172a1ec-9b42-40d1-b7e4-47e7166a9521 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:44.583783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:44.656154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172a1ec-9b42-40d1-b7e4-47e7166a9521 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:44.691378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c41b4-2894-47ab-9cbe-543d4696cdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:44.692121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c41b4-2894-47ab-9cbe-543d4696cdb2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:44.692473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c41b4-2894-47ab-9cbe-543d4696cdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:44.692842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c41b4-2894-47ab-9cbe-543d4696cdb2 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:44.693624 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c41b4-2894-47ab-9cbe-543d4696cdb2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:44.694002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c41b4-2894-47ab-9cbe-543d4696cdb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:44.695435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c41b4-2894-47ab-9cbe-543d4696cdb2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:44.705558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 481/967] 10.222.0.45 () {56 vars in 1122 bytes} [Tue Jul 5 15:51:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:44.706733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0381be6-d381-44c8-a648-b96482541cb5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:44.805859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0381be6-d381-44c8-a648-b96482541cb5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:44.840043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711e24f-e3b7-4d3e-a80d-b4a4c9b38f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:44.840882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711e24f-e3b7-4d3e-a80d-b4a4c9b38f5f None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:44.841266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711e24f-e3b7-4d3e-a80d-b4a4c9b38f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:44.841639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711e24f-e3b7-4d3e-a80d-b4a4c9b38f5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:44.842496 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711e24f-e3b7-4d3e-a80d-b4a4c9b38f5f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:44.843952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711e24f-e3b7-4d3e-a80d-b4a4c9b38f5f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:44.939527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 487/968] 10.222.0.45 () {60 vars in 1144 bytes} [Tue Jul 5 15:51:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:51:45.889598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c395-7e4d-4746-85ad-c3afe5694876 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:45.890167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c395-7e4d-4746-85ad-c3afe5694876 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:45.890167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c395-7e4d-4746-85ad-c3afe5694876 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:45.891517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 482/969] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:45.896750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d526538-c419-4a69-9745-196e81c29e5d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:45.897170 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d526538-c419-4a69-9745-196e81c29e5d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:45.897331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d526538-c419-4a69-9745-196e81c29e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:45.914323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6d526538-c419-4a69-9745-196e81c29e5d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:45.915486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6d526538-c419-4a69-9745-196e81c29e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:45.950502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:45.988735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 488/970] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:46.011218 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c395-7e4d-4746-85ad-c3afe5694876 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.032933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8eddb-7e44-41e6-85f8-c13f0189b196 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.033242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8eddb-7e44-41e6-85f8-c13f0189b196 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.033500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8eddb-7e44-41e6-85f8-c13f0189b196 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.035043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 489/971] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:51:46.039818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafa7d3-ee9e-450e-aba4-761f17a9adbf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.040114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafa7d3-ee9e-450e-aba4-761f17a9adbf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.040396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafa7d3-ee9e-450e-aba4-761f17a9adbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.058669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bbafa7d3-ee9e-450e-aba4-761f17a9adbf None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:46.059884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bbafa7d3-ee9e-450e-aba4-761f17a9adbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:46.109716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:46.121033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c395-7e4d-4746-85ad-c3afe5694876 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:46.129201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 490/972] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:46.151446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafa7d3-ee9e-450e-aba4-761f17a9adbf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.155328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d6a69-2cff-4d9b-918b-4193dba64dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:46.156307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d6a69-2cff-4d9b-918b-4193dba64dbb None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.156733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d6a69-2cff-4d9b-918b-4193dba64dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.157723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d6a69-2cff-4d9b-918b-4193dba64dbb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.168163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d6a69-2cff-4d9b-918b-4193dba64dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-760d6a69-2cff-4d9b-918b-4193dba64dbb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:46.177010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:46.178099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:51:46.179111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:46.181146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 483/973] 10.222.0.45 () {56 vars in 1120 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:46.186729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d6a69-2cff-4d9b-918b-4193dba64dbb None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.251587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:46.255101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafa7d3-ee9e-450e-aba4-761f17a9adbf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:46.290620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b44ed-9d6f-420b-bb19-6a079e00fdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:46.291420 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b44ed-9d6f-420b-bb19-6a079e00fdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.292055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b44ed-9d6f-420b-bb19-6a079e00fdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.292055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b44ed-9d6f-420b-bb19-6a079e00fdf4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.292615 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b44ed-9d6f-420b-bb19-6a079e00fdf4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:46.294023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b44ed-9d6f-420b-bb19-6a079e00fdf4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:46.300253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d6a69-2cff-4d9b-918b-4193dba64dbb None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-227b44ed-9d6f-420b-bb19-6a079e00fdf4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:46.306377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:46.307334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:46.308165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 491/974] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:46.313480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b44ed-9d6f-420b-bb19-6a079e00fdf4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.337035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3adcc-788e-42e7-a93e-a2b6520d2f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:46.337769 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3adcc-788e-42e7-a93e-a2b6520d2f85 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.338118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3adcc-788e-42e7-a93e-a2b6520d2f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.338475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3adcc-788e-42e7-a93e-a2b6520d2f85 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.339216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3adcc-788e-42e7-a93e-a2b6520d2f85 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:46.339525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3adcc-788e-42e7-a93e-a2b6520d2f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:46.340898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3adcc-788e-42e7-a93e-a2b6520d2f85 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:46.350263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 484/975] 10.222.0.45 () {56 vars in 1115 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:46.355570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3adcc-788e-42e7-a93e-a2b6520d2f85 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.372452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:46.423636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b44ed-9d6f-420b-bb19-6a079e00fdf4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:46.458751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c248d-ab19-4710-832f-d222cffaa3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:46.459413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c248d-ab19-4710-832f-d222cffaa3de None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.459693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c248d-ab19-4710-832f-d222cffaa3de None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.459967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c248d-ab19-4710-832f-d222cffaa3de None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.460658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c248d-ab19-4710-832f-d222cffaa3de None admin] RBAC: Adding query filter params (name=compute) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:46.460999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c248d-ab19-4710-832f-d222cffaa3de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:46.462363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c248d-ab19-4710-832f-d222cffaa3de None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:46.472760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 492/976] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:46.472924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3adcc-788e-42e7-a93e-a2b6520d2f85 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:46.474602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c248d-ab19-4710-832f-d222cffaa3de None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.503378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:46.509303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601bd74-ad84-471d-be20-ea5112c78158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:46.510093 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601bd74-ad84-471d-be20-ea5112c78158 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.510506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601bd74-ad84-471d-be20-ea5112c78158 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.513568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601bd74-ad84-471d-be20-ea5112c78158 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.522386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601bd74-ad84-471d-be20-ea5112c78158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:46.523823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601bd74-ad84-471d-be20-ea5112c78158 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-6601bd74-ad84-471d-be20-ea5112c78158 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:46.545764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:46.546714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:46.552482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:46.552482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:46.552482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:46.552482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:46.552482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 485/977] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:46.552804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601bd74-ad84-471d-be20-ea5112c78158 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.585719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c248d-ab19-4710-832f-d222cffaa3de None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:46.623099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390c500-4c65-4f09-b2dc-4cab10063e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:46.623761 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390c500-4c65-4f09-b2dc-4cab10063e89 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.624042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390c500-4c65-4f09-b2dc-4cab10063e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.624314 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390c500-4c65-4f09-b2dc-4cab10063e89 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.625060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390c500-4c65-4f09-b2dc-4cab10063e89 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:46.625361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390c500-4c65-4f09-b2dc-4cab10063e89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:46.626079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:46.627280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390c500-4c65-4f09-b2dc-4cab10063e89 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:46.632500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 493/978] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:46.638331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390c500-4c65-4f09-b2dc-4cab10063e89 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.665549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601bd74-ad84-471d-be20-ea5112c78158 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:46.703665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06200e-2c0c-4af8-8d32-2de2e925a92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:46.704872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06200e-2c0c-4af8-8d32-2de2e925a92a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.705164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06200e-2c0c-4af8-8d32-2de2e925a92a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.705597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06200e-2c0c-4af8-8d32-2de2e925a92a None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.706799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06200e-2c0c-4af8-8d32-2de2e925a92a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:46.707127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06200e-2c0c-4af8-8d32-2de2e925a92a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:46.708492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06200e-2c0c-4af8-8d32-2de2e925a92a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:46.721849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 486/979] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:46.728151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06200e-2c0c-4af8-8d32-2de2e925a92a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.736918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:46.748149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390c500-4c65-4f09-b2dc-4cab10063e89 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:46.785709 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a362c-2428-4491-ac76-60ededce49b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:46.787249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a362c-2428-4491-ac76-60ededce49b2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.787438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a362c-2428-4491-ac76-60ededce49b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.787683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a362c-2428-4491-ac76-60ededce49b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.788478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a362c-2428-4491-ac76-60ededce49b2 None admin] RBAC: Adding query filter params (interface=public, service_id=ac0f057b46d8469ca7ccc9cadf5d5adb, region_id=RegionOne) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:46.788818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a362c-2428-4491-ac76-60ededce49b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:46.790500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a362c-2428-4491-ac76-60ededce49b2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:46.800804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 494/980] 10.222.0.45 () {56 vars in 1251 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/endpoints?service_id=ac0f057b46d8469ca7ccc9cadf5d5adb&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:46.801640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a362c-2428-4491-ac76-60ededce49b2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.831393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06200e-2c0c-4af8-8d32-2de2e925a92a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:46.857978 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:46.867139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d1808-8431-4c91-bd91-bae8ac2ea46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:46.867982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d1808-8431-4c91-bd91-bae8ac2ea46e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.868269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d1808-8431-4c91-bd91-bae8ac2ea46e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.868544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d1808-8431-4c91-bd91-bae8ac2ea46e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.876261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d1808-8431-4c91-bd91-bae8ac2ea46e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:46.877737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d1808-8431-4c91-bd91-bae8ac2ea46e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-7f0d1808-8431-4c91-bd91-bae8ac2ea46e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:46.894112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:46.895643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:46.896652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 487/981] 10.222.0.45 () {56 vars in 1117 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:46.901114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d1808-8431-4c91-bd91-bae8ac2ea46e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:46.917132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a362c-2428-4491-ac76-60ededce49b2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:46.957292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3122ba-e507-4aa3-a168-b9de03fb6167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:46.957881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3122ba-e507-4aa3-a168-b9de03fb6167 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.958178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3122ba-e507-4aa3-a168-b9de03fb6167 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.958443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3122ba-e507-4aa3-a168-b9de03fb6167 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.959139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3122ba-e507-4aa3-a168-b9de03fb6167 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:46.960516 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3122ba-e507-4aa3-a168-b9de03fb6167 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:46.966918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 495/982] 10.222.0.45 () {56 vars in 1087 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:51:46.982075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60dfab2-6212-4cfe-b6c8-1ab352a7d202 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.982868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60dfab2-6212-4cfe-b6c8-1ab352a7d202 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.983341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60dfab2-6212-4cfe-b6c8-1ab352a7d202 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:46.985698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 496/983] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:51:46.992098 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ea6a5-2c04-412a-9d08-8072cdb359e8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:46.992386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ea6a5-2c04-412a-9d08-8072cdb359e8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:46.992675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ea6a5-2c04-412a-9d08-8072cdb359e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.011762 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a98ea6a5-2c04-412a-9d08-8072cdb359e8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:47.013022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a98ea6a5-2c04-412a-9d08-8072cdb359e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:47.028428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:47.031896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d1808-8431-4c91-bd91-bae8ac2ea46e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:47.072186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b95261-74ae-4b8f-a348-bfa1f40c103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:47.073109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b95261-74ae-4b8f-a348-bfa1f40c103f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:47.073745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b95261-74ae-4b8f-a348-bfa1f40c103f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:47.074032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b95261-74ae-4b8f-a348-bfa1f40c103f None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.074891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b95261-74ae-4b8f-a348-bfa1f40c103f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:47.075214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b95261-74ae-4b8f-a348-bfa1f40c103f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:47.077282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b95261-74ae-4b8f-a348-bfa1f40c103f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:47.092448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 497/984] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:47.100802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 488/985] 10.222.0.45 () {56 vars in 1149 bytes} [Tue Jul 5 15:51:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:47.105437 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ea6a5-2c04-412a-9d08-8072cdb359e8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:47.118712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b95261-74ae-4b8f-a348-bfa1f40c103f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:47.167718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:47.214881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ea6a5-2c04-412a-9d08-8072cdb359e8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:47.233416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b95261-74ae-4b8f-a348-bfa1f40c103f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:47.257986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a828dc-b889-417e-bc0f-1a00b5c60377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:47.258641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a828dc-b889-417e-bc0f-1a00b5c60377 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:47.258921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a828dc-b889-417e-bc0f-1a00b5c60377 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:47.259193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a828dc-b889-417e-bc0f-1a00b5c60377 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.278456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:47.278659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01e4a1-ca19-4e18-9f71-dbb13a71e21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:47.279264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01e4a1-ca19-4e18-9f71-dbb13a71e21c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:47.279543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01e4a1-ca19-4e18-9f71-dbb13a71e21c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:47.280039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01e4a1-ca19-4e18-9f71-dbb13a71e21c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.280465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01e4a1-ca19-4e18-9f71-dbb13a71e21c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:47.281873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01e4a1-ca19-4e18-9f71-dbb13a71e21c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-4d01e4a1-ca19-4e18-9f71-dbb13a71e21c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:47.295308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:47.296604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:47.298088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:47.302039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 489/986] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:47.303246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01e4a1-ca19-4e18-9f71-dbb13a71e21c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:47.341513 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a828dc-b889-417e-bc0f-1a00b5c60377 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:47.343203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a828dc-b889-417e-bc0f-1a00b5c60377 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:47.410389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01e4a1-ca19-4e18-9f71-dbb13a71e21c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:47.419803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:47.446439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509de75-440c-47db-8404-59672e60f55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:47.447093 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509de75-440c-47db-8404-59672e60f55a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:47.447372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509de75-440c-47db-8404-59672e60f55a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:47.447643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509de75-440c-47db-8404-59672e60f55a None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.448350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509de75-440c-47db-8404-59672e60f55a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:47.448653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509de75-440c-47db-8404-59672e60f55a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:47.449780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 498/987] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:47 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:51:47.450032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509de75-440c-47db-8404-59672e60f55a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:47.455369 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a828dc-b889-417e-bc0f-1a00b5c60377 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:47.455571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 490/988] 10.222.0.45 () {56 vars in 1122 bytes} [Tue Jul 5 15:51:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:47.461023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509de75-440c-47db-8404-59672e60f55a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:47.512062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:47.564982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509de75-440c-47db-8404-59672e60f55a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:47.565281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a828dc-b889-417e-bc0f-1a00b5c60377 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:47.605310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba726153-3e05-4b39-a1d8-a854ddaada2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:47.606243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba726153-3e05-4b39-a1d8-a854ddaada2b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:47.606725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba726153-3e05-4b39-a1d8-a854ddaada2b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:47.607014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba726153-3e05-4b39-a1d8-a854ddaada2b None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.607733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba726153-3e05-4b39-a1d8-a854ddaada2b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:47.607835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb5870-938b-4abd-b161-ea9b7227b88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:47.608037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba726153-3e05-4b39-a1d8-a854ddaada2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:47.608582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb5870-938b-4abd-b161-ea9b7227b88b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:47.609088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb5870-938b-4abd-b161-ea9b7227b88b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:47.609187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb5870-938b-4abd-b161-ea9b7227b88b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.610563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba726153-3e05-4b39-a1d8-a854ddaada2b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:47.623679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 491/989] 10.222.0.45 () {56 vars in 1122 bytes} [Tue Jul 5 15:51:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:47.623868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb5870-938b-4abd-b161-ea9b7227b88b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:47.623868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb5870-938b-4abd-b161-ea9b7227b88b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:47.625668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba726153-3e05-4b39-a1d8-a854ddaada2b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-eabb5870-938b-4abd-b161-ea9b7227b88b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:47.641082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:47.642082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:47.642975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:47.642975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:47.642975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:47.642975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:47.642975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 499/990] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:47.647901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb5870-938b-4abd-b161-ea9b7227b88b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:47.665221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:47.731795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba726153-3e05-4b39-a1d8-a854ddaada2b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:47.753360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:47.756803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb5870-938b-4abd-b161-ea9b7227b88b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:47.769148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a9b1-ed49-4745-a797-78ac03474e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:47.769148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a9b1-ed49-4745-a797-78ac03474e82 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:47.769148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a9b1-ed49-4745-a797-78ac03474e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:47.769148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a9b1-ed49-4745-a797-78ac03474e82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.769760 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193a9b1-ed49-4745-a797-78ac03474e82 None admin] RBAC: Adding query filter params (interface=admin, service_id=7512bb97a93a41d5a7af17bbddb1ebd5, region_id=RegionOne) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:47.770080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193a9b1-ed49-4745-a797-78ac03474e82 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:47.771444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193a9b1-ed49-4745-a797-78ac03474e82 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:47.776910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 492/991] 10.222.0.45 () {56 vars in 1249 bytes} [Tue Jul 5 15:51:47 2022] GET /identity/v3/endpoints?service_id=7512bb97a93a41d5a7af17bbddb1ebd5&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:47.783217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a9b1-ed49-4745-a797-78ac03474e82 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:47.793348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73086772-eded-4f88-9462-8250f15a1b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:47.794010 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73086772-eded-4f88-9462-8250f15a1b27 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:47.794298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73086772-eded-4f88-9462-8250f15a1b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:47.794577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73086772-eded-4f88-9462-8250f15a1b27 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.795299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73086772-eded-4f88-9462-8250f15a1b27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:47.795605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73086772-eded-4f88-9462-8250f15a1b27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:47.796997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73086772-eded-4f88-9462-8250f15a1b27 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:47.808637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 500/992] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:47.814018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73086772-eded-4f88-9462-8250f15a1b27 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:47.888823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:47.892225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a9b1-ed49-4745-a797-78ac03474e82 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:47.926280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73086772-eded-4f88-9462-8250f15a1b27 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:47.933009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81a55a-0c08-435f-a4a7-5b8277eebd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:47.933806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81a55a-0c08-435f-a4a7-5b8277eebd67 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:47.934191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81a55a-0c08-435f-a4a7-5b8277eebd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:47.934752 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81a55a-0c08-435f-a4a7-5b8277eebd67 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.935357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81a55a-0c08-435f-a4a7-5b8277eebd67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:47.937391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81a55a-0c08-435f-a4a7-5b8277eebd67 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:47.948921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 493/993] 10.222.0.45 () {56 vars in 1087 bytes} [Tue Jul 5 15:51:47 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:51:47.974253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2dd1d4-39ea-4c46-a84d-9d2f1ba7dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:47.975205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2dd1d4-39ea-4c46-a84d-9d2f1ba7dce3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:47.975572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2dd1d4-39ea-4c46-a84d-9d2f1ba7dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:47.975944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2dd1d4-39ea-4c46-a84d-9d2f1ba7dce3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:47.984340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2dd1d4-39ea-4c46-a84d-9d2f1ba7dce3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:47.986090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2dd1d4-39ea-4c46-a84d-9d2f1ba7dce3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-cb2dd1d4-39ea-4c46-a84d-9d2f1ba7dce3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:48.001800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:48.002770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:48.003686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:48.003686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:48.003686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:48.003686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:48.003686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:48.003686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:48.003686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:48.003686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:48.005113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 501/994] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:48.009127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81a55a-0c08-435f-a4a7-5b8277eebd67 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:48.120617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81a55a-0c08-435f-a4a7-5b8277eebd67 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:48.156480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06398a63-d08f-4e63-9195-c63d89d38407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:48.157691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06398a63-d08f-4e63-9195-c63d89d38407 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:48.157978 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06398a63-d08f-4e63-9195-c63d89d38407 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:48.158317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06398a63-d08f-4e63-9195-c63d89d38407 None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:48.159102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06398a63-d08f-4e63-9195-c63d89d38407 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:48.159465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06398a63-d08f-4e63-9195-c63d89d38407 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:48.160966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06398a63-d08f-4e63-9195-c63d89d38407 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:48.173555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 494/995] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:48.179587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2dd1d4-39ea-4c46-a84d-9d2f1ba7dce3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:48.202077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:48.297429 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2dd1d4-39ea-4c46-a84d-9d2f1ba7dce3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:48.333352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805d293-b4a4-4cc8-bea3-082bb741dac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:48.334220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805d293-b4a4-4cc8-bea3-082bb741dac2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:48.334677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805d293-b4a4-4cc8-bea3-082bb741dac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:48.335103 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805d293-b4a4-4cc8-bea3-082bb741dac2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:48.336071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805d293-b4a4-4cc8-bea3-082bb741dac2 None admin] RBAC: Adding query filter params (role.id=450b100797c74a58b92e58120d0c8387, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=7245effe463b45dfa946ca2eda2e0c98) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:48.336533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805d293-b4a4-4cc8-bea3-082bb741dac2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:48.338125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805d293-b4a4-4cc8-bea3-082bb741dac2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:48.344794 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 502/996] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:48 2022] GET /identity/v3/role_assignments?user.id=7245effe463b45dfa946ca2eda2e0c98&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=450b100797c74a58b92e58120d0c8387 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:49.059516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb15ef-6466-4b3d-910b-7ee2990dfa86 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:49.060046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb15ef-6466-4b3d-910b-7ee2990dfa86 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:49.060046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb15ef-6466-4b3d-910b-7ee2990dfa86 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:49.061593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 495/997] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:49.067597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2207-8544-4f3a-9274-8a90cbafe00e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:49.068146 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2207-8544-4f3a-9274-8a90cbafe00e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:49.068576 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2207-8544-4f3a-9274-8a90cbafe00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:49.086506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d9ea2207-8544-4f3a-9274-8a90cbafe00e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:49.087706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d9ea2207-8544-4f3a-9274-8a90cbafe00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:49.169210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 503/998] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:49.192955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb15ef-6466-4b3d-910b-7ee2990dfa86 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:49.222365 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:49.310724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb15ef-6466-4b3d-910b-7ee2990dfa86 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:49.347010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a28b2e-4a55-4799-92c8-a9cfe8bdcf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:49.347799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a28b2e-4a55-4799-92c8-a9cfe8bdcf9f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:49.348079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a28b2e-4a55-4799-92c8-a9cfe8bdcf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:49.348463 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a28b2e-4a55-4799-92c8-a9cfe8bdcf9f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:49.349136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a28b2e-4a55-4799-92c8-a9cfe8bdcf9f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:49.350490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a28b2e-4a55-4799-92c8-a9cfe8bdcf9f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-d9a28b2e-4a55-4799-92c8-a9cfe8bdcf9f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:49.362895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:49.363906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:49.370045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 496/999] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:49.370853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2207-8544-4f3a-9274-8a90cbafe00e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:49.386161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:49.480550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2207-8544-4f3a-9274-8a90cbafe00e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:49.515780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f1250-d9a2-415a-b6b2-4295de38305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:49.517039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f1250-d9a2-415a-b6b2-4295de38305b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:49.517324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f1250-d9a2-415a-b6b2-4295de38305b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:49.517595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f1250-d9a2-415a-b6b2-4295de38305b None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:49.518316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f1250-d9a2-415a-b6b2-4295de38305b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:49.518619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f1250-d9a2-415a-b6b2-4295de38305b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:49.520032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f1250-d9a2-415a-b6b2-4295de38305b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:49.530243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 504/1000] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:49.532361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a28b2e-4a55-4799-92c8-a9cfe8bdcf9f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:49.619548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:49.632345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a28b2e-4a55-4799-92c8-a9cfe8bdcf9f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:49.666447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f93f1c-9de3-4e4c-bf24-e16f7100e2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:49.667113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f93f1c-9de3-4e4c-bf24-e16f7100e2fc None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:49.667394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f93f1c-9de3-4e4c-bf24-e16f7100e2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:49.667666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f93f1c-9de3-4e4c-bf24-e16f7100e2fc None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:49.668367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f93f1c-9de3-4e4c-bf24-e16f7100e2fc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:49.668670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f93f1c-9de3-4e4c-bf24-e16f7100e2fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:49.670074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f93f1c-9de3-4e4c-bf24-e16f7100e2fc None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:49.679538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 497/1001] 10.222.0.45 () {56 vars in 1112 bytes} [Tue Jul 5 15:51:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:49.680718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f1250-d9a2-415a-b6b2-4295de38305b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:49.759855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:49.782547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f1250-d9a2-415a-b6b2-4295de38305b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:49.817817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db77ec-23b3-45da-92a2-2c10ab6c75e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:49.818478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db77ec-23b3-45da-92a2-2c10ab6c75e4 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:49.818765 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db77ec-23b3-45da-92a2-2c10ab6c75e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:49.819044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db77ec-23b3-45da-92a2-2c10ab6c75e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:49.819800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db77ec-23b3-45da-92a2-2c10ab6c75e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:49.821206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db77ec-23b3-45da-92a2-2c10ab6c75e4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:49.924503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 505/1002] 10.222.0.45 () {60 vars in 1144 bytes} [Tue Jul 5 15:51:49 2022] POST /identity/v3/endpoints => generated 334 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:51:50.136250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547816-b425-42c9-895a-f315d7fc3d7b None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:50.136825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547816-b425-42c9-895a-f315d7fc3d7b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:50.136825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547816-b425-42c9-895a-f315d7fc3d7b None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:50.138190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 498/1003] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:50.144285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf27ad2-4b88-44cd-a460-00c9c48157a4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:50.144711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf27ad2-4b88-44cd-a460-00c9c48157a4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:50.144882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf27ad2-4b88-44cd-a460-00c9c48157a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:50.162520 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5bf27ad2-4b88-44cd-a460-00c9c48157a4 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:50.163747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5bf27ad2-4b88-44cd-a460-00c9c48157a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:50.242672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 506/1004] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:50 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:50.265028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba547816-b425-42c9-895a-f315d7fc3d7b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:50.378391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba547816-b425-42c9-895a-f315d7fc3d7b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:50.412609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7841e-cd16-4506-8819-a49e8cbae20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:50.413439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7841e-cd16-4506-8819-a49e8cbae20b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:50.413727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7841e-cd16-4506-8819-a49e8cbae20b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:50.414003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7841e-cd16-4506-8819-a49e8cbae20b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:50.414649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7841e-cd16-4506-8819-a49e8cbae20b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:50.416008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7841e-cd16-4506-8819-a49e8cbae20b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-e2f7841e-cd16-4506-8819-a49e8cbae20b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:50.428005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:50.429295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:50.430151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 499/1005] 10.222.0.45 () {56 vars in 1126 bytes} [Tue Jul 5 15:51:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:50.435464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2d8a6-0b0c-4586-a701-5821e92152e2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:50.435464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2d8a6-0b0c-4586-a701-5821e92152e2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:50.435464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2d8a6-0b0c-4586-a701-5821e92152e2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:50.436406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 507/1006] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:50.437071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7841e-cd16-4506-8819-a49e8cbae20b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:50.441564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9898e7-56f7-4a0d-8f08-9522a66d1030 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:50.442074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9898e7-56f7-4a0d-8f08-9522a66d1030 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:50.442529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9898e7-56f7-4a0d-8f08-9522a66d1030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:50.457343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:50.460147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4a9898e7-56f7-4a0d-8f08-9522a66d1030 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:50.461555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4a9898e7-56f7-4a0d-8f08-9522a66d1030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:50.522685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:50.536857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 508/1007] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:50 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:50.543415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7841e-cd16-4506-8819-a49e8cbae20b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:50.556513 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9898e7-56f7-4a0d-8f08-9522a66d1030 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:50.580511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c271f-3637-44e8-ad4a-63ed94e1f626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:50.581292 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c271f-3637-44e8-ad4a-63ed94e1f626 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:50.581648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c271f-3637-44e8-ad4a-63ed94e1f626 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:50.581993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c271f-3637-44e8-ad4a-63ed94e1f626 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:50.582782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c271f-3637-44e8-ad4a-63ed94e1f626 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:50.583163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c271f-3637-44e8-ad4a-63ed94e1f626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:50.584642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c271f-3637-44e8-ad4a-63ed94e1f626 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:50.594153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 500/1008] 10.222.0.45 () {56 vars in 1122 bytes} [Tue Jul 5 15:51:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:50.595347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c271f-3637-44e8-ad4a-63ed94e1f626 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:50.662528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9898e7-56f7-4a0d-8f08-9522a66d1030 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:50.679608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:50.700051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c271f-3637-44e8-ad4a-63ed94e1f626 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:50.702158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab12a2-9af9-405b-80c6-8edce5598380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:50.703033 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab12a2-9af9-405b-80c6-8edce5598380 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:50.703426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab12a2-9af9-405b-80c6-8edce5598380 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:50.704360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab12a2-9af9-405b-80c6-8edce5598380 None admin] PATH_INFO: `/v3/services/image` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:50.705177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab12a2-9af9-405b-80c6-8edce5598380 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:50.706660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab12a2-9af9-405b-80c6-8edce5598380 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-b0ab12a2-9af9-405b-80c6-8edce5598380 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:50.718712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:50.719899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:51:50.720979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:50.722093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 509/1009] 10.222.0.45 () {56 vars in 1105 bytes} [Tue Jul 5 15:51:50 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:50.728141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab12a2-9af9-405b-80c6-8edce5598380 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:50.740369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b89c8-200d-4e2d-a105-5c9d0dba5d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:50.741193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b89c8-200d-4e2d-a105-5c9d0dba5d7a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:50.741579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b89c8-200d-4e2d-a105-5c9d0dba5d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:50.741971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b89c8-200d-4e2d-a105-5c9d0dba5d7a None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:50.742793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b89c8-200d-4e2d-a105-5c9d0dba5d7a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:50.743204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b89c8-200d-4e2d-a105-5c9d0dba5d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:50.744676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b89c8-200d-4e2d-a105-5c9d0dba5d7a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:50.754860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 501/1010] 10.222.0.45 () {56 vars in 1122 bytes} [Tue Jul 5 15:51:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:50.756568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b89c8-200d-4e2d-a105-5c9d0dba5d7a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:50.791631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:50.839177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab12a2-9af9-405b-80c6-8edce5598380 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:50.859301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b89c8-200d-4e2d-a105-5c9d0dba5d7a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:50.878989 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40eb65-75c5-496f-a426-a1ce414adc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:50.879658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40eb65-75c5-496f-a426-a1ce414adc29 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:50.880361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40eb65-75c5-496f-a426-a1ce414adc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:50.880649 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40eb65-75c5-496f-a426-a1ce414adc29 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:50.881677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40eb65-75c5-496f-a426-a1ce414adc29 None admin] RBAC: Adding query filter params (name=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:50.881997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40eb65-75c5-496f-a426-a1ce414adc29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:50.884862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40eb65-75c5-496f-a426-a1ce414adc29 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:50.897567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 510/1011] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:50.897730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c05a1-0688-4f59-8c27-163b6dc6fa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:50.897730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c05a1-0688-4f59-8c27-163b6dc6fa0e None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:50.897730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c05a1-0688-4f59-8c27-163b6dc6fa0e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:50.897730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c05a1-0688-4f59-8c27-163b6dc6fa0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:50.898432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c05a1-0688-4f59-8c27-163b6dc6fa0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:50.899214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40eb65-75c5-496f-a426-a1ce414adc29 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:50.899785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c05a1-0688-4f59-8c27-163b6dc6fa0e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:50.960891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 502/1012] 10.222.0.45 () {60 vars in 1144 bytes} [Tue Jul 5 15:51:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:51.023555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40eb65-75c5-496f-a426-a1ce414adc29 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:51.043068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:51.069543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f41946-0e70-49c0-9512-6252fc7113b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:51.070366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f41946-0e70-49c0-9512-6252fc7113b2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:51.070718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f41946-0e70-49c0-9512-6252fc7113b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:51.071059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f41946-0e70-49c0-9512-6252fc7113b2 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:51.071849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f41946-0e70-49c0-9512-6252fc7113b2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:51.072226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f41946-0e70-49c0-9512-6252fc7113b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:51.073724 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f41946-0e70-49c0-9512-6252fc7113b2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:51.084809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 511/1013] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:52.050683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a1d0f-0568-4ecf-934c-18f090013aad None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:52.051229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a1d0f-0568-4ecf-934c-18f090013aad None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:52.051229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a1d0f-0568-4ecf-934c-18f090013aad None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:52.052705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 503/1014] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:52.057590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e59e8a-cc13-4b60-8b35-fd3d25105b8f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:52.057965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e59e8a-cc13-4b60-8b35-fd3d25105b8f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:52.058146 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e59e8a-cc13-4b60-8b35-fd3d25105b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:52.074857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-43e59e8a-cc13-4b60-8b35-fd3d25105b8f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:52.076042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-43e59e8a-cc13-4b60-8b35-fd3d25105b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:52.147081 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 512/1015] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:52 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:52.170424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a1d0f-0568-4ecf-934c-18f090013aad None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:52.174967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:52.276010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a1d0f-0568-4ecf-934c-18f090013aad None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:52.311971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21389e1b-1ae9-4218-acc8-1ef27758917b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:52.312800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21389e1b-1ae9-4218-acc8-1ef27758917b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:52.313095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21389e1b-1ae9-4218-acc8-1ef27758917b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:52.313376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21389e1b-1ae9-4218-acc8-1ef27758917b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:52.323713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21389e1b-1ae9-4218-acc8-1ef27758917b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-21389e1b-1ae9-4218-acc8-1ef27758917b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:52.332328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:52.333664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:52.334514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 504/1016] 10.222.0.45 () {56 vars in 1120 bytes} [Tue Jul 5 15:51:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:52.341173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e59e8a-cc13-4b60-8b35-fd3d25105b8f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:52.440811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e59e8a-cc13-4b60-8b35-fd3d25105b8f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:52.450076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:52.475992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f249da-23b0-457e-aed6-81178896b2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:52.476941 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f249da-23b0-457e-aed6-81178896b2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:52.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f249da-23b0-457e-aed6-81178896b2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:52.477514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f249da-23b0-457e-aed6-81178896b2e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:52.478264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f249da-23b0-457e-aed6-81178896b2e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:52.478575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f249da-23b0-457e-aed6-81178896b2e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:52.479948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f249da-23b0-457e-aed6-81178896b2e8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:52.492542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 513/1017] 10.222.0.45 () {56 vars in 1115 bytes} [Tue Jul 5 15:51:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:52.494044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21389e1b-1ae9-4218-acc8-1ef27758917b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:52.558283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:52.597891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21389e1b-1ae9-4218-acc8-1ef27758917b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:52.631757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be323707-c230-4e5a-8518-fd0a75f97fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:52.632598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be323707-c230-4e5a-8518-fd0a75f97fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:52.632917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be323707-c230-4e5a-8518-fd0a75f97fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:52.633197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be323707-c230-4e5a-8518-fd0a75f97fd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:52.644513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be323707-c230-4e5a-8518-fd0a75f97fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:52.645988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be323707-c230-4e5a-8518-fd0a75f97fd5 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-be323707-c230-4e5a-8518-fd0a75f97fd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:52.663035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:52.663949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:52.665280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:52.665280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:52.665280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:52.665280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:52.668958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 505/1018] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:52.670124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f249da-23b0-457e-aed6-81178896b2e8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:52.772832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f249da-23b0-457e-aed6-81178896b2e8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:52.806076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fea87-4883-4b9a-9c0e-ec72d5ddca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:52.806739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fea87-4883-4b9a-9c0e-ec72d5ddca27 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:52.807026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fea87-4883-4b9a-9c0e-ec72d5ddca27 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:52.807304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fea87-4883-4b9a-9c0e-ec72d5ddca27 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:52.808051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fea87-4883-4b9a-9c0e-ec72d5ddca27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:52.808364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fea87-4883-4b9a-9c0e-ec72d5ddca27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:52.809769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fea87-4883-4b9a-9c0e-ec72d5ddca27 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:52.822783 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 514/1019] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:52.828177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be323707-c230-4e5a-8518-fd0a75f97fd5 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:52.931225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be323707-c230-4e5a-8518-fd0a75f97fd5 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:52.950066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:52.968788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d32db-6a39-44b3-91e5-f830580c5044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:52.969645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d32db-6a39-44b3-91e5-f830580c5044 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:52.969935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d32db-6a39-44b3-91e5-f830580c5044 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:52.970216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d32db-6a39-44b3-91e5-f830580c5044 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:52.978227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d32db-6a39-44b3-91e5-f830580c5044 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:52.979886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d32db-6a39-44b3-91e5-f830580c5044 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-148d32db-6a39-44b3-91e5-f830580c5044 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:52.994704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:52.995746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:52.996704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:52.996704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:52.996704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:52.996704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:52.996704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:52.996704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:52.996704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:52.996704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:52.996704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:52.996704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:52.997978 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 506/1020] 10.222.0.45 () {56 vars in 1093 bytes} [Tue Jul 5 15:51:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:53.002737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fea87-4883-4b9a-9c0e-ec72d5ddca27 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:53.016196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086a12-a4d8-4925-aceb-1426ba6b53de None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.016528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086a12-a4d8-4925-aceb-1426ba6b53de None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.016797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086a12-a4d8-4925-aceb-1426ba6b53de None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.018183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 507/1021] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:51:53.023535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b53d4-5156-42c8-acaf-a24ceff69a76 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.023826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b53d4-5156-42c8-acaf-a24ceff69a76 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.024103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b53d4-5156-42c8-acaf-a24ceff69a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.035330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:53.041433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e91b53d4-5156-42c8-acaf-a24ceff69a76 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:53.042389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e91b53d4-5156-42c8-acaf-a24ceff69a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:53.109311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fea87-4883-4b9a-9c0e-ec72d5ddca27 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:53.113712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 508/1022] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:53 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:53.115088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7185e-af3d-4ea0-8b67-325578fae39d None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.115387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7185e-af3d-4ea0-8b67-325578fae39d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.115663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7185e-af3d-4ea0-8b67-325578fae39d None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.120573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 509/1023] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:53.121846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637f746-70f3-4f8c-a776-bcf4d6a6e806 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.122150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637f746-70f3-4f8c-a776-bcf4d6a6e806 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.122422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637f746-70f3-4f8c-a776-bcf4d6a6e806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.139765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e637f746-70f3-4f8c-a776-bcf4d6a6e806 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:53.140984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e637f746-70f3-4f8c-a776-bcf4d6a6e806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:53.149389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289db6c1-b4be-4aac-8be9-367e00d12aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:53.150061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289db6c1-b4be-4aac-8be9-367e00d12aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.150347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289db6c1-b4be-4aac-8be9-367e00d12aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.150623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289db6c1-b4be-4aac-8be9-367e00d12aa6 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.151453 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289db6c1-b4be-4aac-8be9-367e00d12aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:53.151765 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289db6c1-b4be-4aac-8be9-367e00d12aa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:53.153228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289db6c1-b4be-4aac-8be9-367e00d12aa6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:53.170377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 515/1024] 10.222.0.45 () {56 vars in 1133 bytes} [Tue Jul 5 15:51:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:53.171209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289db6c1-b4be-4aac-8be9-367e00d12aa6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:53.210461 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 510/1025] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:53 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:53.210461 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637f746-70f3-4f8c-a776-bcf4d6a6e806 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:53.249173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:53.260328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:53.272260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289db6c1-b4be-4aac-8be9-367e00d12aa6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:53.306037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d239db-b05d-461a-8cfa-fb73ca69c575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:53.306828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d239db-b05d-461a-8cfa-fb73ca69c575 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.307423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d239db-b05d-461a-8cfa-fb73ca69c575 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.307756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d239db-b05d-461a-8cfa-fb73ca69c575 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.308950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d239db-b05d-461a-8cfa-fb73ca69c575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:53.310382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d239db-b05d-461a-8cfa-fb73ca69c575 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:53.311991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 516/1026] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:53.312730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d239db-b05d-461a-8cfa-fb73ca69c575 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:53.316878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637f746-70f3-4f8c-a776-bcf4d6a6e806 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:53.351282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e09db-5a39-4632-b8d7-63c2a621caa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:53.352187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e09db-5a39-4632-b8d7-63c2a621caa2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.352471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e09db-5a39-4632-b8d7-63c2a621caa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.352748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e09db-5a39-4632-b8d7-63c2a621caa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.412362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d239db-b05d-461a-8cfa-fb73ca69c575 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:53.418570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e09db-5a39-4632-b8d7-63c2a621caa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:53.420015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e09db-5a39-4632-b8d7-63c2a621caa2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:53.446848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e25bb5-b257-4374-a4c6-e51c63be074e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:53.447495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e25bb5-b257-4374-a4c6-e51c63be074e None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.447772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e25bb5-b257-4374-a4c6-e51c63be074e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.448042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e25bb5-b257-4374-a4c6-e51c63be074e None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.448751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e25bb5-b257-4374-a4c6-e51c63be074e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:53.450109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e25bb5-b257-4374-a4c6-e51c63be074e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:53.455142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:53.517639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 511/1027] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:53 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:51:53.518357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e09db-5a39-4632-b8d7-63c2a621caa2 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:53.520924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 517/1028] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:51:53 2022] POST /identity/v3/services => generated 242 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:53.523332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e25bb5-b257-4374-a4c6-e51c63be074e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:53.544557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:53.636311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e09db-5a39-4632-b8d7-63c2a621caa2 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:53.649138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e25bb5-b257-4374-a4c6-e51c63be074e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:53.672970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e7e7f-c759-4475-a9d7-c20ca60316cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:53.673787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e7e7f-c759-4475-a9d7-c20ca60316cc None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.674068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e7e7f-c759-4475-a9d7-c20ca60316cc None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.674346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e7e7f-c759-4475-a9d7-c20ca60316cc None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.675084 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e7e7f-c759-4475-a9d7-c20ca60316cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:53.676508 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e7e7f-c759-4475-a9d7-c20ca60316cc None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:53.681840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:53.688229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f762d-fadb-4088-93fb-8b87e5072600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:53.689068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f762d-fadb-4088-93fb-8b87e5072600 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.689353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f762d-fadb-4088-93fb-8b87e5072600 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.689629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f762d-fadb-4088-93fb-8b87e5072600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.700337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f762d-fadb-4088-93fb-8b87e5072600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:53.702046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f762d-fadb-4088-93fb-8b87e5072600 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-849f762d-fadb-4088-93fb-8b87e5072600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:53.720373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:53.721550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:53.722450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:53.722450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:53.722450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:53.722450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:53.722450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 518/1029] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:53.727620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f762d-fadb-4088-93fb-8b87e5072600 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:53.770172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 512/1030] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:51:53 2022] POST /identity/v3/projects => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:53.796394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:53.839408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f762d-fadb-4088-93fb-8b87e5072600 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:53.877804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115acf25-7f07-4209-85e5-e102be7c3bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:53.878482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115acf25-7f07-4209-85e5-e102be7c3bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:53.878762 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115acf25-7f07-4209-85e5-e102be7c3bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:53.879035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115acf25-7f07-4209-85e5-e102be7c3bc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:53.879767 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115acf25-7f07-4209-85e5-e102be7c3bc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:53.880075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115acf25-7f07-4209-85e5-e102be7c3bc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:53.881506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115acf25-7f07-4209-85e5-e102be7c3bc9 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:53.894686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 519/1031] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:53.900194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e7e7f-c759-4475-a9d7-c20ca60316cc None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:54.006039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e7e7f-c759-4475-a9d7-c20ca60316cc None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:54.039703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16014772-2a2f-4901-a1ed-26d98bc801fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:54.040416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16014772-2a2f-4901-a1ed-26d98bc801fd None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:54.040698 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16014772-2a2f-4901-a1ed-26d98bc801fd None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:54.040992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16014772-2a2f-4901-a1ed-26d98bc801fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:54.048166 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16014772-2a2f-4901-a1ed-26d98bc801fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:54.049634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16014772-2a2f-4901-a1ed-26d98bc801fd None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-16014772-2a2f-4901-a1ed-26d98bc801fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:54.063877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:54.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:54.065892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:54.065892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:54.065892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:54.065892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:54.065892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:54.065892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:54.065892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:54.065892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:54.065892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 513/1032] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:54.070820 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115acf25-7f07-4209-85e5-e102be7c3bc9 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:54.175686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115acf25-7f07-4209-85e5-e102be7c3bc9 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:54.198483 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:54.209380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a305d2f-59a4-4f62-937a-1f558a2b4a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:54.214101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a305d2f-59a4-4f62-937a-1f558a2b4a82 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:54.214101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a305d2f-59a4-4f62-937a-1f558a2b4a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:54.214101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a305d2f-59a4-4f62-937a-1f558a2b4a82 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:54.214101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a305d2f-59a4-4f62-937a-1f558a2b4a82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:54.214101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a305d2f-59a4-4f62-937a-1f558a2b4a82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:54.214101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a305d2f-59a4-4f62-937a-1f558a2b4a82 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:54.231237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 520/1033] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:54.232644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16014772-2a2f-4901-a1ed-26d98bc801fd None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:54.284130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:54.338703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16014772-2a2f-4901-a1ed-26d98bc801fd None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:54.373945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfc4ab-a107-4e05-8761-a6ff488ed394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:54.374597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc4ab-a107-4e05-8761-a6ff488ed394 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:54.374883 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc4ab-a107-4e05-8761-a6ff488ed394 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:54.375160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc4ab-a107-4e05-8761-a6ff488ed394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:54.376028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfc4ab-a107-4e05-8761-a6ff488ed394 None admin] RBAC: Adding query filter params (role.id=450b100797c74a58b92e58120d0c8387, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=ef95ab7b924d456195a8761e894d911b) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:54.376346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfc4ab-a107-4e05-8761-a6ff488ed394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:54.377801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfc4ab-a107-4e05-8761-a6ff488ed394 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:54.389014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 514/1034] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:54 2022] GET /identity/v3/role_assignments?user.id=ef95ab7b924d456195a8761e894d911b&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=450b100797c74a58b92e58120d0c8387 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:55.557857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88067-3afc-4c01-bb60-c6e507182247 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:55.558348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88067-3afc-4c01-bb60-c6e507182247 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:55.558348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88067-3afc-4c01-bb60-c6e507182247 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:55.559843 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 521/1035] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:51:55.565422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965e91f-fbfc-43ca-bab7-870c53ba9ab2 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:55.565834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965e91f-fbfc-43ca-bab7-870c53ba9ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:55.566004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965e91f-fbfc-43ca-bab7-870c53ba9ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:55.583146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b965e91f-fbfc-43ca-bab7-870c53ba9ab2 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:55.584333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b965e91f-fbfc-43ca-bab7-870c53ba9ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:55.653835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 515/1036] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:55.676984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc88067-3afc-4c01-bb60-c6e507182247 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:55.778088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc88067-3afc-4c01-bb60-c6e507182247 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:55.812559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55a738-c713-4149-ba21-067edac3d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:55.813320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55a738-c713-4149-ba21-067edac3d897 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:55.813601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55a738-c713-4149-ba21-067edac3d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:55.813874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55a738-c713-4149-ba21-067edac3d897 None admin] PATH_INFO: `/v3/services/image` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:55.814527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55a738-c713-4149-ba21-067edac3d897 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:55.815890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55a738-c713-4149-ba21-067edac3d897 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8c55a738-c713-4149-ba21-067edac3d897 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:55.827472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:55.831707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:55.832562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 522/1037] 10.222.0.45 () {56 vars in 1105 bytes} [Tue Jul 5 15:51:55 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:55.834606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965e91f-fbfc-43ca-bab7-870c53ba9ab2 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:55.835301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff86da-27da-48a9-a676-c37f070bfeff None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:55.835596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff86da-27da-48a9-a676-c37f070bfeff None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:55.835879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff86da-27da-48a9-a676-c37f070bfeff None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:55.837300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 523/1038] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:55.843551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e041f4-f3d2-46bd-aba2-01203dc95967 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:55.843845 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e041f4-f3d2-46bd-aba2-01203dc95967 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:55.844656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e041f4-f3d2-46bd-aba2-01203dc95967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:55.862717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-19e041f4-f3d2-46bd-aba2-01203dc95967 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:55.865860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-19e041f4-f3d2-46bd-aba2-01203dc95967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:55.887790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:55.909556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:55.934932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 524/1039] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:55.950476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965e91f-fbfc-43ca-bab7-870c53ba9ab2 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:55.958892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e041f4-f3d2-46bd-aba2-01203dc95967 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:55.987175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8593a-31a3-4d81-9d5f-7509b55eb340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:55.987840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8593a-31a3-4d81-9d5f-7509b55eb340 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:55.988127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8593a-31a3-4d81-9d5f-7509b55eb340 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:55.988399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8593a-31a3-4d81-9d5f-7509b55eb340 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:55.989138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8593a-31a3-4d81-9d5f-7509b55eb340 None admin] RBAC: Adding query filter params (name=image) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:55.989448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8593a-31a3-4d81-9d5f-7509b55eb340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:55.990806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d8593a-31a3-4d81-9d5f-7509b55eb340 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:55.995960 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 516/1040] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:56.001416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8593a-31a3-4d81-9d5f-7509b55eb340 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:56.062354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e041f4-f3d2-46bd-aba2-01203dc95967 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:56.101082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c891d-7d17-4729-8207-cbbe87f83f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:56.102039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c891d-7d17-4729-8207-cbbe87f83f7e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:56.102320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c891d-7d17-4729-8207-cbbe87f83f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:56.102586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c891d-7d17-4729-8207-cbbe87f83f7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:56.103755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c891d-7d17-4729-8207-cbbe87f83f7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:56.105204 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c891d-7d17-4729-8207-cbbe87f83f7e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:56.106813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 525/1041] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:56.107950 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8593a-31a3-4d81-9d5f-7509b55eb340 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:56.112459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c891d-7d17-4729-8207-cbbe87f83f7e None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:56.119087 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:56.146971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba2b21-9030-4a50-952c-1c69bed9c9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:56.147628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba2b21-9030-4a50-952c-1c69bed9c9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:56.147910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba2b21-9030-4a50-952c-1c69bed9c9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:56.148184 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba2b21-9030-4a50-952c-1c69bed9c9c9 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:56.148919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba2b21-9030-4a50-952c-1c69bed9c9c9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:56.149233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba2b21-9030-4a50-952c-1c69bed9c9c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:56.151020 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba2b21-9030-4a50-952c-1c69bed9c9c9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:56.161338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 517/1042] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:56.162205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba2b21-9030-4a50-952c-1c69bed9c9c9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:56.169342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:56.226516 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c891d-7d17-4729-8207-cbbe87f83f7e None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:56.261313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ddb06-7b0d-496e-9897-9ef1c476e410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:56.261986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ddb06-7b0d-496e-9897-9ef1c476e410 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:56.262275 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ddb06-7b0d-496e-9897-9ef1c476e410 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:56.262545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ddb06-7b0d-496e-9897-9ef1c476e410 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:56.263249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ddb06-7b0d-496e-9897-9ef1c476e410 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:56.264676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ddb06-7b0d-496e-9897-9ef1c476e410 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:56.266346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba2b21-9030-4a50-952c-1c69bed9c9c9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:56.302122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6aca5-74a0-441a-8223-6d31af57e6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:56.302778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6aca5-74a0-441a-8223-6d31af57e6ad None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:56.303069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6aca5-74a0-441a-8223-6d31af57e6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:56.303345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6aca5-74a0-441a-8223-6d31af57e6ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:56.304194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6aca5-74a0-441a-8223-6d31af57e6ad None admin] RBAC: Adding query filter params (interface=public, service_id=c3643bf1300541249e7a7540b9c0dde7, region_id=RegionOne) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:56.304507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6aca5-74a0-441a-8223-6d31af57e6ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:56.305906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6aca5-74a0-441a-8223-6d31af57e6ad None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:56.310964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 518/1043] 10.222.0.45 () {56 vars in 1251 bytes} [Tue Jul 5 15:51:56 2022] GET /identity/v3/endpoints?service_id=c3643bf1300541249e7a7540b9c0dde7&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:56.316394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6aca5-74a0-441a-8223-6d31af57e6ad None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:56.350366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:56.367181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 526/1044] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:51:56 2022] POST /identity/v3/users => generated 285 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:56.402364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db778da4-6e3d-4935-a66c-4cc0bfa2c80a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:56.402666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db778da4-6e3d-4935-a66c-4cc0bfa2c80a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:56.402947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db778da4-6e3d-4935-a66c-4cc0bfa2c80a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:56.408821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 527/1045] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:51:56.410658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799b5ce-b287-4f0e-95b2-a7f1935817fc None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:56.411019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799b5ce-b287-4f0e-95b2-a7f1935817fc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:56.411400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799b5ce-b287-4f0e-95b2-a7f1935817fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:56.423511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6aca5-74a0-441a-8223-6d31af57e6ad None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:56.429276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c799b5ce-b287-4f0e-95b2-a7f1935817fc None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:56.430514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c799b5ce-b287-4f0e-95b2-a7f1935817fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:56.462557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb5db4-b8b9-4299-b4ea-a75bb4bb12c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:56.463257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb5db4-b8b9-4299-b4ea-a75bb4bb12c8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:56.463537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb5db4-b8b9-4299-b4ea-a75bb4bb12c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:56.463807 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb5db4-b8b9-4299-b4ea-a75bb4bb12c8 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:56.464507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb5db4-b8b9-4299-b4ea-a75bb4bb12c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:56.465914 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb5db4-b8b9-4299-b4ea-a75bb4bb12c8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:56.480810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 519/1046] 10.222.0.45 () {56 vars in 1087 bytes} [Tue Jul 5 15:51:56 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:51:56.514520 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 528/1047] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:56.538598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb5db4-b8b9-4299-b4ea-a75bb4bb12c8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:56.646952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:56.650727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb5db4-b8b9-4299-b4ea-a75bb4bb12c8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:56.684655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06023d6c-23e8-4f01-927c-80e9bae4a0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:56.685560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06023d6c-23e8-4f01-927c-80e9bae4a0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:56.685839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06023d6c-23e8-4f01-927c-80e9bae4a0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:56.686108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06023d6c-23e8-4f01-927c-80e9bae4a0b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:56.697628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06023d6c-23e8-4f01-927c-80e9bae4a0b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-06023d6c-23e8-4f01-927c-80e9bae4a0b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:56.706311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:56.707235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:56.713224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 520/1048] 10.222.0.45 () {56 vars in 1120 bytes} [Tue Jul 5 15:51:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:56.715703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799b5ce-b287-4f0e-95b2-a7f1935817fc None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:56.748197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:56.819809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799b5ce-b287-4f0e-95b2-a7f1935817fc None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:56.854788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d82b6-93ef-4b51-a16a-ac454fce5136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:56.855464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d82b6-93ef-4b51-a16a-ac454fce5136 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:56.855751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d82b6-93ef-4b51-a16a-ac454fce5136 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:56.856030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d82b6-93ef-4b51-a16a-ac454fce5136 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:56.856801 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d82b6-93ef-4b51-a16a-ac454fce5136 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:56.857115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d82b6-93ef-4b51-a16a-ac454fce5136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:56.858539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d82b6-93ef-4b51-a16a-ac454fce5136 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:56.867610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 529/1049] 10.222.0.45 () {56 vars in 1115 bytes} [Tue Jul 5 15:51:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:56.873268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06023d6c-23e8-4f01-927c-80e9bae4a0b6 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:56.977669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06023d6c-23e8-4f01-927c-80e9bae4a0b6 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:57.012071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c67a09-d644-43a7-9bfa-b95d8862319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:57.012887 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c67a09-d644-43a7-9bfa-b95d8862319d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:57.013171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c67a09-d644-43a7-9bfa-b95d8862319d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:57.013446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c67a09-d644-43a7-9bfa-b95d8862319d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:57.024559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c67a09-d644-43a7-9bfa-b95d8862319d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:57.026131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c67a09-d644-43a7-9bfa-b95d8862319d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-a2c67a09-d644-43a7-9bfa-b95d8862319d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:57.045270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:57.046188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:57.047610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:57.047610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:57.047610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:57.047610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:57.047610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 521/1050] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:57.056798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d82b6-93ef-4b51-a16a-ac454fce5136 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:57.150834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:57.158407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d82b6-93ef-4b51-a16a-ac454fce5136 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:57.192650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f806bd5-08d3-43a4-a715-d75fb817f02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:57.193350 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806bd5-08d3-43a4-a715-d75fb817f02f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:57.193633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806bd5-08d3-43a4-a715-d75fb817f02f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:57.193912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806bd5-08d3-43a4-a715-d75fb817f02f None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:57.194641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f806bd5-08d3-43a4-a715-d75fb817f02f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:57.194947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f806bd5-08d3-43a4-a715-d75fb817f02f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:57.196316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f806bd5-08d3-43a4-a715-d75fb817f02f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:57.212330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 530/1051] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:57.213697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c67a09-d644-43a7-9bfa-b95d8862319d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:57.227647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:57.321598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c67a09-d644-43a7-9bfa-b95d8862319d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:57.355167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403eb43-a8ed-4a49-8e8c-137f75ef9e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:57.356222 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403eb43-a8ed-4a49-8e8c-137f75ef9e6c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:57.356709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403eb43-a8ed-4a49-8e8c-137f75ef9e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:57.357218 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403eb43-a8ed-4a49-8e8c-137f75ef9e6c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:57.365221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403eb43-a8ed-4a49-8e8c-137f75ef9e6c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:57.366863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403eb43-a8ed-4a49-8e8c-137f75ef9e6c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-1403eb43-a8ed-4a49-8e8c-137f75ef9e6c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:51:57.380797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:57.381736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:57.382681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 522/1052] 10.222.0.45 () {56 vars in 1093 bytes} [Tue Jul 5 15:51:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:57.389398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f806bd5-08d3-43a4-a715-d75fb817f02f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:57.492723 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f806bd5-08d3-43a4-a715-d75fb817f02f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:57.525872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93dc53-3afc-4656-a13f-0705b846f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:57.526546 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93dc53-3afc-4656-a13f-0705b846f043 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:57.526931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93dc53-3afc-4656-a13f-0705b846f043 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:57.527216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93dc53-3afc-4656-a13f-0705b846f043 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:57.528053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93dc53-3afc-4656-a13f-0705b846f043 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:57.528368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93dc53-3afc-4656-a13f-0705b846f043 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:57.529820 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e93dc53-3afc-4656-a13f-0705b846f043 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:57.549154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 531/1053] 10.222.0.45 () {56 vars in 1133 bytes} [Tue Jul 5 15:51:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:57.554830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403eb43-a8ed-4a49-8e8c-137f75ef9e6c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:57.627294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:57.658744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403eb43-a8ed-4a49-8e8c-137f75ef9e6c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:57.692393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac1339-d407-4c43-bbd4-ed035c696812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:57.693219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac1339-d407-4c43-bbd4-ed035c696812 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:57.693507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac1339-d407-4c43-bbd4-ed035c696812 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:57.693784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac1339-d407-4c43-bbd4-ed035c696812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:57.704822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac1339-d407-4c43-bbd4-ed035c696812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:57.706263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac1339-d407-4c43-bbd4-ed035c696812 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-32ac1339-d407-4c43-bbd4-ed035c696812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:51:57.723686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:57.724564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:57.730457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:51:57.730457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:51:57.730457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:51:57.730457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:57.730457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 523/1054] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:51:57.730744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93dc53-3afc-4656-a13f-0705b846f043 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:57.745580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:57.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93dc53-3afc-4656-a13f-0705b846f043 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:57.866736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616b79b-1afd-41c6-ae12-69d99053ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:57.867394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616b79b-1afd-41c6-ae12-69d99053ce9f None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:57.867678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616b79b-1afd-41c6-ae12-69d99053ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:57.867954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616b79b-1afd-41c6-ae12-69d99053ce9f None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:57.868672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616b79b-1afd-41c6-ae12-69d99053ce9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:57.869009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616b79b-1afd-41c6-ae12-69d99053ce9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:57.870428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616b79b-1afd-41c6-ae12-69d99053ce9f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:57.886620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 532/1055] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:57.887882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac1339-d407-4c43-bbd4-ed035c696812 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:57.988324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac1339-d407-4c43-bbd4-ed035c696812 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:58.011042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:58.023334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd66bc-8caa-41bc-83d1-8f74a21a18ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:58.024676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd66bc-8caa-41bc-83d1-8f74a21a18ae None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:58.024676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd66bc-8caa-41bc-83d1-8f74a21a18ae None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:58.024676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd66bc-8caa-41bc-83d1-8f74a21a18ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:58.031336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd66bc-8caa-41bc-83d1-8f74a21a18ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:58.032803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd66bc-8caa-41bc-83d1-8f74a21a18ae None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-bebd66bc-8caa-41bc-83d1-8f74a21a18ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:51:58.047483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:58.048389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:58.053869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:51:58.053869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:51:58.053869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:51:58.053869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:51:58.053869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:51:58.053869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:51:58.053869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:51:58.053869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:58.053869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 524/1056] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:51:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:58.056691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616b79b-1afd-41c6-ae12-69d99053ce9f None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:58.144231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:58.177121 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616b79b-1afd-41c6-ae12-69d99053ce9f None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:58.211614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e304fc-8e6c-4729-9117-2cce9a9014d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:58.212279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e304fc-8e6c-4729-9117-2cce9a9014d3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:58.212561 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e304fc-8e6c-4729-9117-2cce9a9014d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:58.212867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e304fc-8e6c-4729-9117-2cce9a9014d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:58.213658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e304fc-8e6c-4729-9117-2cce9a9014d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:58.213967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e304fc-8e6c-4729-9117-2cce9a9014d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:58.215389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e304fc-8e6c-4729-9117-2cce9a9014d3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:58.227925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 533/1057] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:51:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:58.233648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd66bc-8caa-41bc-83d1-8f74a21a18ae None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:58.347193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd66bc-8caa-41bc-83d1-8f74a21a18ae None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:58.381782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc4536-1691-4c7e-aecb-9a3a9cb3512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:58.382648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc4536-1691-4c7e-aecb-9a3a9cb3512d None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:58.382928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc4536-1691-4c7e-aecb-9a3a9cb3512d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:58.383224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc4536-1691-4c7e-aecb-9a3a9cb3512d None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/ef95ab7b924d456195a8761e894d911b/roles/450b100797c74a58b92e58120d0c8387` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:58.385364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc4536-1691-4c7e-aecb-9a3a9cb3512d None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=ef95ab7b924d456195a8761e894d911b, role_id=450b100797c74a58b92e58120d0c8387)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:58.386799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc4536-1691-4c7e-aecb-9a3a9cb3512d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:58.455117 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 525/1058] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:51:58 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/ef95ab7b924d456195a8761e894d911b/roles/450b100797c74a58b92e58120d0c8387 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:51:58.499727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e65118-d76f-4111-b8e3-75b1d1a44f82 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:58.500643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e65118-d76f-4111-b8e3-75b1d1a44f82 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:58.500919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e65118-d76f-4111-b8e3-75b1d1a44f82 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:58.502544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 534/1059] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:58.508146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220c8ff-9670-4bec-95de-661ead3400a3 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:58.508451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220c8ff-9670-4bec-95de-661ead3400a3 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:58.508718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220c8ff-9670-4bec-95de-661ead3400a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:58.526588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8220c8ff-9670-4bec-95de-661ead3400a3 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:58.527813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8220c8ff-9670-4bec-95de-661ead3400a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:58.566469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:58.568215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd8c4-0593-49ad-8670-68796be57230 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:58.568876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd8c4-0593-49ad-8670-68796be57230 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:58.569166 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd8c4-0593-49ad-8670-68796be57230 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:58.576444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 535/1060] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:51:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:51:58.582790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944773f-4016-4a02-9327-4461a892ac9d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:58.583088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944773f-4016-4a02-9327-4461a892ac9d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:58.583371 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944773f-4016-4a02-9327-4461a892ac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:58.606459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4944773f-4016-4a02-9327-4461a892ac9d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:51:58.607306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4944773f-4016-4a02-9327-4461a892ac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:51:58.611786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 526/1061] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:51:58.635161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220c8ff-9670-4bec-95de-661ead3400a3 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:58.694175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 536/1062] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:51:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:51:58.716733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944773f-4016-4a02-9327-4461a892ac9d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:58.740336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220c8ff-9670-4bec-95de-661ead3400a3 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:58.777840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:58.778738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30318a-e071-4e8f-8c2e-cd8d011045fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:58.779588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30318a-e071-4e8f-8c2e-cd8d011045fd None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:58.779877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30318a-e071-4e8f-8c2e-cd8d011045fd None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:58.780156 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30318a-e071-4e8f-8c2e-cd8d011045fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:58.791186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30318a-e071-4e8f-8c2e-cd8d011045fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-ba30318a-e071-4e8f-8c2e-cd8d011045fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:51:58.799393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:58.800347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:51:58.801244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 527/1063] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:51:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:58.806392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30318a-e071-4e8f-8c2e-cd8d011045fd None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:58.821953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944773f-4016-4a02-9327-4461a892ac9d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:58.838902 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:58.867007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f958acc-f767-4d6c-82bf-cfecd7aba29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:58.867795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f958acc-f767-4d6c-82bf-cfecd7aba29b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:58.868096 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f958acc-f767-4d6c-82bf-cfecd7aba29b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:58.868372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f958acc-f767-4d6c-82bf-cfecd7aba29b None admin] PATH_INFO: `/v3/services/image` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:58.869059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f958acc-f767-4d6c-82bf-cfecd7aba29b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:58.870432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f958acc-f767-4d6c-82bf-cfecd7aba29b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-6f958acc-f767-4d6c-82bf-cfecd7aba29b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:51:58.883137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:51:58.884197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:51:58.889647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 537/1064] 10.222.0.45 () {56 vars in 1105 bytes} [Tue Jul 5 15:51:58 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:51:58.890950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f958acc-f767-4d6c-82bf-cfecd7aba29b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:58.914999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30318a-e071-4e8f-8c2e-cd8d011045fd None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:58.955385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e684e-79e0-4939-80ee-97900ff753e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:58.956489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e684e-79e0-4939-80ee-97900ff753e9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:58.956798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e684e-79e0-4939-80ee-97900ff753e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:58.957423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e684e-79e0-4939-80ee-97900ff753e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:58.958209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e684e-79e0-4939-80ee-97900ff753e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:58.958652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e684e-79e0-4939-80ee-97900ff753e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:58.960661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e684e-79e0-4939-80ee-97900ff753e9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:58.972649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 528/1065] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:51:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:58.978750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e684e-79e0-4939-80ee-97900ff753e9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:59.004968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f958acc-f767-4d6c-82bf-cfecd7aba29b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:59.033728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:59.042592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078acf4-ec4f-4d9b-94d8-780341ae31e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:59.048954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078acf4-ec4f-4d9b-94d8-780341ae31e0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:59.048954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078acf4-ec4f-4d9b-94d8-780341ae31e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:59.048954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078acf4-ec4f-4d9b-94d8-780341ae31e0 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:59.048954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078acf4-ec4f-4d9b-94d8-780341ae31e0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:59.048954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078acf4-ec4f-4d9b-94d8-780341ae31e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:59.048954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078acf4-ec4f-4d9b-94d8-780341ae31e0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:59.056035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 538/1066] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:59.056805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078acf4-ec4f-4d9b-94d8-780341ae31e0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:59.086798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:59.096871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e684e-79e0-4939-80ee-97900ff753e9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:59.136689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a75f25-bdbf-4301-b492-26b5195d8fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:59.137709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a75f25-bdbf-4301-b492-26b5195d8fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:59.137993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a75f25-bdbf-4301-b492-26b5195d8fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:59.138267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a75f25-bdbf-4301-b492-26b5195d8fa0 None admin] PATH_INFO: `/v3/users/04940a6aa89f418e88842dd3a13ed374` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:59.154465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a75f25-bdbf-4301-b492-26b5195d8fa0 None admin] RBAC: Authorizing `identity:get_user(user_id=04940a6aa89f418e88842dd3a13ed374)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:59.155926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a75f25-bdbf-4301-b492-26b5195d8fa0 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:59.163018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 529/1067] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:51:58 2022] GET /identity/v3/users/04940a6aa89f418e88842dd3a13ed374 => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:59.163153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078acf4-ec4f-4d9b-94d8-780341ae31e0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:59.164020 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a75f25-bdbf-4301-b492-26b5195d8fa0 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:59.197049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b872fa8-7431-46d2-9a68-125010b9cbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:59.197717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b872fa8-7431-46d2-9a68-125010b9cbea None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:59.198028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b872fa8-7431-46d2-9a68-125010b9cbea None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:59.198308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b872fa8-7431-46d2-9a68-125010b9cbea None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:59.199026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b872fa8-7431-46d2-9a68-125010b9cbea None admin] RBAC: Adding query filter params (type=image) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:59.199335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b872fa8-7431-46d2-9a68-125010b9cbea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:59.200713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b872fa8-7431-46d2-9a68-125010b9cbea None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:59.206323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 539/1068] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:51:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:51:59.211923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b872fa8-7431-46d2-9a68-125010b9cbea None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:59.273845 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:59.274447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a75f25-bdbf-4301-b492-26b5195d8fa0 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:59.312947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1121f8-242b-46d6-a30d-68d8c0afd0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:59.313596 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1121f8-242b-46d6-a30d-68d8c0afd0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:59.313880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1121f8-242b-46d6-a30d-68d8c0afd0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:59.314155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1121f8-242b-46d6-a30d-68d8c0afd0a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:59.320834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b872fa8-7431-46d2-9a68-125010b9cbea None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:59.338807 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:59.358842 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167af06-196a-4ac1-8fb5-9fee8af413da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:59.359479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167af06-196a-4ac1-8fb5-9fee8af413da None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:59.359760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167af06-196a-4ac1-8fb5-9fee8af413da None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:59.360036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167af06-196a-4ac1-8fb5-9fee8af413da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:59.360793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167af06-196a-4ac1-8fb5-9fee8af413da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:59.362157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167af06-196a-4ac1-8fb5-9fee8af413da None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:59.387450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1121f8-242b-46d6-a30d-68d8c0afd0a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:59.388944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1121f8-242b-46d6-a30d-68d8c0afd0a0 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:59.441737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 540/1069] 10.222.0.45 () {60 vars in 1144 bytes} [Tue Jul 5 15:51:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:51:59.501208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 530/1070] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:51:59 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:51:59.502791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167af06-196a-4ac1-8fb5-9fee8af413da None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:59.624289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167af06-196a-4ac1-8fb5-9fee8af413da None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:59.633871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:59.663564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870083ed-7535-4891-90d1-ab5723c71bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:59.664720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870083ed-7535-4891-90d1-ab5723c71bdc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:59.665288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870083ed-7535-4891-90d1-ab5723c71bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:59.665747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870083ed-7535-4891-90d1-ab5723c71bdc None admin] PATH_INFO: `/v3/projects/3d22f09fa68241c08b79554641f15ddd` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:59.667094 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870083ed-7535-4891-90d1-ab5723c71bdc None admin] RBAC: Authorizing `identity:get_project(project_id=3d22f09fa68241c08b79554641f15ddd)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:59.668714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870083ed-7535-4891-90d1-ab5723c71bdc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:59.671068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 541/1071] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:51:59 2022] GET /identity/v3/projects/3d22f09fa68241c08b79554641f15ddd => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:51:59.677240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1121f8-242b-46d6-a30d-68d8c0afd0a0 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:51:59.733613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:51:59.782601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1121f8-242b-46d6-a30d-68d8c0afd0a0 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:51:59.816323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5529579-61a8-4eb3-8488-d4fe311c2c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:51:59.817011 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5529579-61a8-4eb3-8488-d4fe311c2c7a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:51:59.817300 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5529579-61a8-4eb3-8488-d4fe311c2c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:51:59.817579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5529579-61a8-4eb3-8488-d4fe311c2c7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:51:59.818409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5529579-61a8-4eb3-8488-d4fe311c2c7a None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=3d22f09fa68241c08b79554641f15ddd, user.id=04940a6aa89f418e88842dd3a13ed374) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:51:59.818725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5529579-61a8-4eb3-8488-d4fe311c2c7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:51:59.820142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5529579-61a8-4eb3-8488-d4fe311c2c7a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:51:59.832840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 531/1072] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:51:59 2022] GET /identity/v3/role_assignments?user.id=04940a6aa89f418e88842dd3a13ed374&scope.project.id=3d22f09fa68241c08b79554641f15ddd&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:52:00.504448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ceb4aa-c756-4953-a8fa-aaaee75ecf3e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:00.504979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ceb4aa-c756-4953-a8fa-aaaee75ecf3e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:00.505055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ceb4aa-c756-4953-a8fa-aaaee75ecf3e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:00.506425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 542/1073] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:00.511448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953e95-172d-4f17-955a-8a8468ded136 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:00.511818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953e95-172d-4f17-955a-8a8468ded136 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:00.512168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953e95-172d-4f17-955a-8a8468ded136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:00.529066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-46953e95-172d-4f17-955a-8a8468ded136 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:00.530249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-46953e95-172d-4f17-955a-8a8468ded136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:00.604876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 532/1074] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:00.626307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ceb4aa-c756-4953-a8fa-aaaee75ecf3e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:00.727606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ceb4aa-c756-4953-a8fa-aaaee75ecf3e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:00.761473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1399c22-4d24-4026-b186-e17dbbd3589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:00.762271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1399c22-4d24-4026-b186-e17dbbd3589a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:00.762549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1399c22-4d24-4026-b186-e17dbbd3589a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:00.762821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1399c22-4d24-4026-b186-e17dbbd3589a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:00.772890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1399c22-4d24-4026-b186-e17dbbd3589a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e1399c22-4d24-4026-b186-e17dbbd3589a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:52:00.780876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:00.781831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:52:00.782693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:00.784838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 543/1075] 10.222.0.45 () {56 vars in 1120 bytes} [Tue Jul 5 15:52:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:00.788040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46953e95-172d-4f17-955a-8a8468ded136 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:00.836728 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:00.890651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46953e95-172d-4f17-955a-8a8468ded136 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:00.928036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0b6e0-4912-4a0a-a490-fbf32ee9957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:00.928733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0b6e0-4912-4a0a-a490-fbf32ee9957c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:00.929038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0b6e0-4912-4a0a-a490-fbf32ee9957c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:00.929316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0b6e0-4912-4a0a-a490-fbf32ee9957c None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:00.930079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0b6e0-4912-4a0a-a490-fbf32ee9957c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:00.930386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0b6e0-4912-4a0a-a490-fbf32ee9957c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:00.931780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0b6e0-4912-4a0a-a490-fbf32ee9957c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:00.945203 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 533/1076] 10.222.0.45 () {56 vars in 1115 bytes} [Tue Jul 5 15:52:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:00.946479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1399c22-4d24-4026-b186-e17dbbd3589a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:00.982795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:01.051040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1399c22-4d24-4026-b186-e17dbbd3589a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:01.084466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c904c3-47f8-48b2-8638-79abf42cd0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:01.085271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c904c3-47f8-48b2-8638-79abf42cd0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:01.085551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c904c3-47f8-48b2-8638-79abf42cd0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:01.085826 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c904c3-47f8-48b2-8638-79abf42cd0e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:01.096283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c904c3-47f8-48b2-8638-79abf42cd0e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:01.097756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c904c3-47f8-48b2-8638-79abf42cd0e0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-51c904c3-47f8-48b2-8638-79abf42cd0e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:52:01.115590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:01.116475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:01.117742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:52:01.117742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:52:01.117742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:52:01.117742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:01.121081 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 544/1077] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:01.122453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0b6e0-4912-4a0a-a490-fbf32ee9957c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:01.232290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0b6e0-4912-4a0a-a490-fbf32ee9957c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:01.241463 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:01.266611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318814a-7977-4074-971c-e0d1e0779cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:01.267284 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318814a-7977-4074-971c-e0d1e0779cbb None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:01.267565 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318814a-7977-4074-971c-e0d1e0779cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:01.267836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318814a-7977-4074-971c-e0d1e0779cbb None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:01.268564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318814a-7977-4074-971c-e0d1e0779cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:01.273205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318814a-7977-4074-971c-e0d1e0779cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:01.275108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318814a-7977-4074-971c-e0d1e0779cbb None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:01.292984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 534/1078] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:01.293948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c904c3-47f8-48b2-8638-79abf42cd0e0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:01.381139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:01.396126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c904c3-47f8-48b2-8638-79abf42cd0e0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:01.431134 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8bd11-3b64-4fa0-818c-2b3b7dbb1491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:01.431919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8bd11-3b64-4fa0-818c-2b3b7dbb1491 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:01.432196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8bd11-3b64-4fa0-818c-2b3b7dbb1491 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:01.432468 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8bd11-3b64-4fa0-818c-2b3b7dbb1491 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:01.439598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8bd11-3b64-4fa0-818c-2b3b7dbb1491 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:01.441061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8bd11-3b64-4fa0-818c-2b3b7dbb1491 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-57e8bd11-3b64-4fa0-818c-2b3b7dbb1491 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:52:01.455749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:01.456682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:01.457658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 545/1079] 10.222.0.45 () {56 vars in 1093 bytes} [Tue Jul 5 15:52:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:01.464151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318814a-7977-4074-971c-e0d1e0779cbb None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:01.507715 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5bc56-6027-4e08-90ce-bfe36046fc04 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:01.508217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5bc56-6027-4e08-90ce-bfe36046fc04 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:01.508217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5bc56-6027-4e08-90ce-bfe36046fc04 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:01.509628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 546/1080] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:52:01.514378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7e90-ae6e-4b6c-a92a-1cfe00bfa4d8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:01.514668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7e90-ae6e-4b6c-a92a-1cfe00bfa4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:01.514939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7e90-ae6e-4b6c-a92a-1cfe00bfa4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:01.532448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f6ff7e90-ae6e-4b6c-a92a-1cfe00bfa4d8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:01.533671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6ff7e90-ae6e-4b6c-a92a-1cfe00bfa4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:01.571694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318814a-7977-4074-971c-e0d1e0779cbb None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:01.602463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 547/1081] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:52:01.611769 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce7459-6675-47b8-95c5-b16489967169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:01.612462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce7459-6675-47b8-95c5-b16489967169 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:01.612742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce7459-6675-47b8-95c5-b16489967169 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:01.613041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce7459-6675-47b8-95c5-b16489967169 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:01.613874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce7459-6675-47b8-95c5-b16489967169 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:01.614185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce7459-6675-47b8-95c5-b16489967169 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:01.615607 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce7459-6675-47b8-95c5-b16489967169 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:01.625737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff7e90-ae6e-4b6c-a92a-1cfe00bfa4d8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:01.635052 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 535/1082] 10.222.0.45 () {56 vars in 1133 bytes} [Tue Jul 5 15:52:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:01.640358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce7459-6675-47b8-95c5-b16489967169 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:01.669431 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:01.711015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:01.734485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff7e90-ae6e-4b6c-a92a-1cfe00bfa4d8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:01.746048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce7459-6675-47b8-95c5-b16489967169 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:01.771760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8f76c-c9b1-4df8-bb5c-6a3fafabc8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:01.772613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8f76c-c9b1-4df8-bb5c-6a3fafabc8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:01.772929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8f76c-c9b1-4df8-bb5c-6a3fafabc8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:01.773208 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8f76c-c9b1-4df8-bb5c-6a3fafabc8e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:01.781879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75811604-58e5-4839-98b4-1b3cb33e2210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:01.782527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75811604-58e5-4839-98b4-1b3cb33e2210 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:01.782810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75811604-58e5-4839-98b4-1b3cb33e2210 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:01.783090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75811604-58e5-4839-98b4-1b3cb33e2210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:01.852775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8f76c-c9b1-4df8-bb5c-6a3fafabc8e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:01.854244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8f76c-c9b1-4df8-bb5c-6a3fafabc8e7 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:01.858502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75811604-58e5-4839-98b4-1b3cb33e2210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:01.859996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75811604-58e5-4839-98b4-1b3cb33e2210 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:01.885947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:01.931851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:01.958237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 548/1083] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:52:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:52:01.958685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 536/1084] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:52:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:52:01.960289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82910fa-8665-4d9a-adb8-7a69371e9a42 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:01.960717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82910fa-8665-4d9a-adb8-7a69371e9a42 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:01.961161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82910fa-8665-4d9a-adb8-7a69371e9a42 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:01.963278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 549/1085] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:01.964497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75811604-58e5-4839-98b4-1b3cb33e2210 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:01.964748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82910fa-8665-4d9a-adb8-7a69371e9a42 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:02.065955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75811604-58e5-4839-98b4-1b3cb33e2210 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.066227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82910fa-8665-4d9a-adb8-7a69371e9a42 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.090524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:02.098971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aab0e7-631d-4a7d-abfc-56ccf18886f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.099400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab5ed9-3682-406c-a8a4-48a323a00c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.099746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab0e7-631d-4a7d-abfc-56ccf18886f4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.100025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab0e7-631d-4a7d-abfc-56ccf18886f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.104356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab5ed9-3682-406c-a8a4-48a323a00c24 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.104600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab0e7-631d-4a7d-abfc-56ccf18886f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.104690 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab5ed9-3682-406c-a8a4-48a323a00c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.104690 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab5ed9-3682-406c-a8a4-48a323a00c24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.111487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aab0e7-631d-4a7d-abfc-56ccf18886f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:02.111650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab5ed9-3682-406c-a8a4-48a323a00c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:02.112975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aab0e7-631d-4a7d-abfc-56ccf18886f4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:02.113074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab5ed9-3682-406c-a8a4-48a323a00c24 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-88aab0e7-631d-4a7d-abfc-56ccf18886f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:52:02.130602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:02.131705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:02.132659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:52:02.132659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:52:02.132659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:52:02.132659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-e3ab5ed9-3682-406c-a8a4-48a323a00c24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:52:02.133276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:02.134068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:02.134967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:52:02.134967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:52:02.134967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:52:02.134967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:02.135186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 537/1086] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:02.135186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e535a0-cddd-4d0d-a297-7f8923e31cff None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.135186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e535a0-cddd-4d0d-a297-7f8923e31cff None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.135186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e535a0-cddd-4d0d-a297-7f8923e31cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.136719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 550/1087] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:02.137543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab5ed9-3682-406c-a8a4-48a323a00c24 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:02.151334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-48e535a0-cddd-4d0d-a297-7f8923e31cff None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:02.152522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-48e535a0-cddd-4d0d-a297-7f8923e31cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:02.164748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:02.219215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 538/1088] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:02.220040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e535a0-cddd-4d0d-a297-7f8923e31cff None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:02.239553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab5ed9-3682-406c-a8a4-48a323a00c24 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.275850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c9034-88e7-4e4c-b80e-d57e0c80f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.276510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c9034-88e7-4e4c-b80e-d57e0c80f885 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.276818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c9034-88e7-4e4c-b80e-d57e0c80f885 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.277094 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c9034-88e7-4e4c-b80e-d57e0c80f885 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.277818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c9034-88e7-4e4c-b80e-d57e0c80f885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:02.278112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c9034-88e7-4e4c-b80e-d57e0c80f885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:02.279478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c9034-88e7-4e4c-b80e-d57e0c80f885 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:02.291909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 551/1089] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:02.292797 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c9034-88e7-4e4c-b80e-d57e0c80f885 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:02.324817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e535a0-cddd-4d0d-a297-7f8923e31cff None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.334587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:02.361168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c06573-e0f4-4db3-86d5-5c259a1abb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.361833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c06573-e0f4-4db3-86d5-5c259a1abb71 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.362412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c06573-e0f4-4db3-86d5-5c259a1abb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.362972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c06573-e0f4-4db3-86d5-5c259a1abb71 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.363720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c06573-e0f4-4db3-86d5-5c259a1abb71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:02.364305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c06573-e0f4-4db3-86d5-5c259a1abb71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:02.365773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c06573-e0f4-4db3-86d5-5c259a1abb71 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:02.377343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 539/1090] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:02.378110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c06573-e0f4-4db3-86d5-5c259a1abb71 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:02.396745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:02.412557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c9034-88e7-4e4c-b80e-d57e0c80f885 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.452333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa63452-1114-4915-8864-9ec942b89fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.453761 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa63452-1114-4915-8864-9ec942b89fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.454045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa63452-1114-4915-8864-9ec942b89fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.454319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa63452-1114-4915-8864-9ec942b89fb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.467156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa63452-1114-4915-8864-9ec942b89fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4fa63452-1114-4915-8864-9ec942b89fb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:52:02.475067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:02.476090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:02.477232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 552/1091] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:52:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:02.482811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa63452-1114-4915-8864-9ec942b89fb6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:02.510811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c06573-e0f4-4db3-86d5-5c259a1abb71 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.549720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ce6ae-1d66-49a4-81ed-15c070894fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.550519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ce6ae-1d66-49a4-81ed-15c070894fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.550803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ce6ae-1d66-49a4-81ed-15c070894fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.551076 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ce6ae-1d66-49a4-81ed-15c070894fd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.557994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ce6ae-1d66-49a4-81ed-15c070894fd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:02.559432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ce6ae-1d66-49a4-81ed-15c070894fd0 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-e17ce6ae-1d66-49a4-81ed-15c070894fd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:52:02.574217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:02.575355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:02.576364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:52:02.576364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:52:02.576364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:52:02.576364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:52:02.576364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:52:02.576364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:52:02.576364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:52:02.576364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:52:02.576364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 540/1092] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:52:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:02.583515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ce6ae-1d66-49a4-81ed-15c070894fd0 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:02.603090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa63452-1114-4915-8864-9ec942b89fb6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.621569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:02.644599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d0299-e9e3-4e69-bb86-d6f793f12059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.645310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d0299-e9e3-4e69-bb86-d6f793f12059 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.645615 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d0299-e9e3-4e69-bb86-d6f793f12059 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.645867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d0299-e9e3-4e69-bb86-d6f793f12059 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.646842 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d0299-e9e3-4e69-bb86-d6f793f12059 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:02.646956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d0299-e9e3-4e69-bb86-d6f793f12059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:02.648285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d0299-e9e3-4e69-bb86-d6f793f12059 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:02.657765 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 553/1093] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:52:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:52:02.664406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d0299-e9e3-4e69-bb86-d6f793f12059 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:02.676282 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:02.706171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ce6ae-1d66-49a4-81ed-15c070894fd0 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.739576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941277e-2182-4332-83b6-35a4349bb62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.740224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941277e-2182-4332-83b6-35a4349bb62b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.740505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941277e-2182-4332-83b6-35a4349bb62b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.740795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941277e-2182-4332-83b6-35a4349bb62b None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.741590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941277e-2182-4332-83b6-35a4349bb62b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:02.741896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941277e-2182-4332-83b6-35a4349bb62b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:02.743306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941277e-2182-4332-83b6-35a4349bb62b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:02.755063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 541/1094] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:52:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:52:02.760893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941277e-2182-4332-83b6-35a4349bb62b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:02.762918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d0299-e9e3-4e69-bb86-d6f793f12059 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.796124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ad43d-6272-45fa-8fd8-ed7fb0e6eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.797549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad43d-6272-45fa-8fd8-ed7fb0e6eb4a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.797832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad43d-6272-45fa-8fd8-ed7fb0e6eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.798105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ad43d-6272-45fa-8fd8-ed7fb0e6eb4a None admin] PATH_INFO: `/v3/users/04940a6aa89f418e88842dd3a13ed374` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.799314 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ad43d-6272-45fa-8fd8-ed7fb0e6eb4a None admin] RBAC: Authorizing `identity:get_user(user_id=04940a6aa89f418e88842dd3a13ed374)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:02.800745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ad43d-6272-45fa-8fd8-ed7fb0e6eb4a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:02.802423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 554/1095] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:02 2022] GET /identity/v3/users/04940a6aa89f418e88842dd3a13ed374 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:02.808080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ad43d-6272-45fa-8fd8-ed7fb0e6eb4a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:02.846566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:02.858059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941277e-2182-4332-83b6-35a4349bb62b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.885823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:02.892977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ae4b-bae5-4715-a781-4ac4c99bc97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.893621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ae4b-bae5-4715-a781-4ac4c99bc97e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.893905 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ae4b-bae5-4715-a781-4ac4c99bc97e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.894186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ae4b-bae5-4715-a781-4ac4c99bc97e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.895010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18ae4b-bae5-4715-a781-4ac4c99bc97e None admin] RBAC: Adding query filter params (role.id=450b100797c74a58b92e58120d0c8387, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=ef95ab7b924d456195a8761e894d911b) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:02.895320 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18ae4b-bae5-4715-a781-4ac4c99bc97e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:02.896747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18ae4b-bae5-4715-a781-4ac4c99bc97e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:02.902713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 542/1096] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:52:02 2022] GET /identity/v3/role_assignments?user.id=ef95ab7b924d456195a8761e894d911b&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=450b100797c74a58b92e58120d0c8387 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:52:02.906261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ad43d-6272-45fa-8fd8-ed7fb0e6eb4a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:02.945432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e5f8b-fcdd-49fb-8605-de554cd5a81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:02.946662 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e5f8b-fcdd-49fb-8605-de554cd5a81a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:02.946948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e5f8b-fcdd-49fb-8605-de554cd5a81a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:02.947229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e5f8b-fcdd-49fb-8605-de554cd5a81a None admin] PATH_INFO: `/v3/projects/3d22f09fa68241c08b79554641f15ddd` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:02.948599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e5f8b-fcdd-49fb-8605-de554cd5a81a None admin] RBAC: Authorizing `identity:get_project(project_id=3d22f09fa68241c08b79554641f15ddd)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:02.950153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e5f8b-fcdd-49fb-8605-de554cd5a81a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:02.952043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 555/1097] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:02 2022] GET /identity/v3/projects/3d22f09fa68241c08b79554641f15ddd => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:52:02.962792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ae4b-bae5-4715-a781-4ac4c99bc97e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:03.071520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ae4b-bae5-4715-a781-4ac4c99bc97e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:03.106230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11b60b-be00-4794-999a-cb36f8421519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:03.107038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b60b-be00-4794-999a-cb36f8421519 None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:03.107324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b60b-be00-4794-999a-cb36f8421519 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:03.107604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b60b-be00-4794-999a-cb36f8421519 None admin] PATH_INFO: `/v3/projects/3d22f09fa68241c08b79554641f15ddd/users/04940a6aa89f418e88842dd3a13ed374/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:03.110148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11b60b-be00-4794-999a-cb36f8421519 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d22f09fa68241c08b79554641f15ddd, user_id=04940a6aa89f418e88842dd3a13ed374, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:03.111570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11b60b-be00-4794-999a-cb36f8421519 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:03.206525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 543/1098] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:52:02 2022] PUT /identity/v3/projects/3d22f09fa68241c08b79554641f15ddd/users/04940a6aa89f418e88842dd3a13ed374/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:52:05.211238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20845b1b-40d6-4f7b-acd4-7ed5477d65e9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:05.211765 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20845b1b-40d6-4f7b-acd4-7ed5477d65e9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:05.211858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20845b1b-40d6-4f7b-acd4-7ed5477d65e9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:05.213182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 556/1099] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:05.218401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef974a9f-45cc-481a-8792-50876286c822 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:05.218840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef974a9f-45cc-481a-8792-50876286c822 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:05.218962 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef974a9f-45cc-481a-8792-50876286c822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:05.237133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-ef974a9f-45cc-481a-8792-50876286c822 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:05.238316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ef974a9f-45cc-481a-8792-50876286c822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:05.245702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:05.303364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 544/1100] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:05.327176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20845b1b-40d6-4f7b-acd4-7ed5477d65e9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:05.426703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20845b1b-40d6-4f7b-acd4-7ed5477d65e9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:05.458426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d892b8-8cc7-4ac1-8c38-21acb4344904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:05.459196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d892b8-8cc7-4ac1-8c38-21acb4344904 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:05.459476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d892b8-8cc7-4ac1-8c38-21acb4344904 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:05.459748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d892b8-8cc7-4ac1-8c38-21acb4344904 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:05.470083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d892b8-8cc7-4ac1-8c38-21acb4344904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-93d892b8-8cc7-4ac1-8c38-21acb4344904 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:52:05.477775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:05.478850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:05.480060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 557/1101] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:05.485362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef974a9f-45cc-481a-8792-50876286c822 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:05.580046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef974a9f-45cc-481a-8792-50876286c822 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:05.613236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e62e6-c454-484c-be30-46e18866c856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:05.613877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e62e6-c454-484c-be30-46e18866c856 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:05.614157 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e62e6-c454-484c-be30-46e18866c856 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:05.614430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e62e6-c454-484c-be30-46e18866c856 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:05.615161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e62e6-c454-484c-be30-46e18866c856 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:05.615465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e62e6-c454-484c-be30-46e18866c856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:05.616833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e62e6-c454-484c-be30-46e18866c856 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:05.624507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 545/1102] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:05.630052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d892b8-8cc7-4ac1-8c38-21acb4344904 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:05.639219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:05.728979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d892b8-8cc7-4ac1-8c38-21acb4344904 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:05.761156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928aacfd-7360-4bbe-80cf-fdaad01aeea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:05.762000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928aacfd-7360-4bbe-80cf-fdaad01aeea2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:05.762287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928aacfd-7360-4bbe-80cf-fdaad01aeea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:05.762566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928aacfd-7360-4bbe-80cf-fdaad01aeea2 None admin] PATH_INFO: `/v3/users/04940a6aa89f418e88842dd3a13ed374` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:05.763743 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928aacfd-7360-4bbe-80cf-fdaad01aeea2 None admin] RBAC: Authorizing `identity:get_user(user_id=04940a6aa89f418e88842dd3a13ed374)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:05.765209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928aacfd-7360-4bbe-80cf-fdaad01aeea2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:05.767332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 558/1103] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:05 2022] GET /identity/v3/users/04940a6aa89f418e88842dd3a13ed374 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:05.773048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e62e6-c454-484c-be30-46e18866c856 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:05.776667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:05.870991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e62e6-c454-484c-be30-46e18866c856 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:05.903401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84a3d-434e-4fba-8a89-2aeb8ebe8793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:05.904035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84a3d-434e-4fba-8a89-2aeb8ebe8793 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:05.904322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84a3d-434e-4fba-8a89-2aeb8ebe8793 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:05.904601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe84a3d-434e-4fba-8a89-2aeb8ebe8793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:05.970051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe84a3d-434e-4fba-8a89-2aeb8ebe8793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:05.971493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe84a3d-434e-4fba-8a89-2aeb8ebe8793 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:05.974024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:06.065999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 546/1104] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:52:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:52:06.071718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928aacfd-7360-4bbe-80cf-fdaad01aeea2 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:06.145402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:06.169240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928aacfd-7360-4bbe-80cf-fdaad01aeea2 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:06.201868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd012c-f11e-4e39-9908-ca86759a7cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:06.202717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd012c-f11e-4e39-9908-ca86759a7cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:06.203523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd012c-f11e-4e39-9908-ca86759a7cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:06.203800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd012c-f11e-4e39-9908-ca86759a7cc2 None admin] PATH_INFO: `/v3/projects/3d22f09fa68241c08b79554641f15ddd` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:06.204997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd012c-f11e-4e39-9908-ca86759a7cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=3d22f09fa68241c08b79554641f15ddd)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:06.206430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd012c-f11e-4e39-9908-ca86759a7cc2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:06.208105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 559/1105] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:06 2022] GET /identity/v3/projects/3d22f09fa68241c08b79554641f15ddd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:06.213894 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84a3d-434e-4fba-8a89-2aeb8ebe8793 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:06.309486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe84a3d-434e-4fba-8a89-2aeb8ebe8793 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:06.311725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:06.345405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf22a27-5380-452a-b478-1763cef1d40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:06.346018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf22a27-5380-452a-b478-1763cef1d40f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:06.346305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf22a27-5380-452a-b478-1763cef1d40f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:06.346583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf22a27-5380-452a-b478-1763cef1d40f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:06.347452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf22a27-5380-452a-b478-1763cef1d40f None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=3d22f09fa68241c08b79554641f15ddd, user.id=04940a6aa89f418e88842dd3a13ed374) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:06.347819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf22a27-5380-452a-b478-1763cef1d40f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:06.349362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf22a27-5380-452a-b478-1763cef1d40f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:06.355945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 547/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:52:06 2022] GET /identity/v3/role_assignments?user.id=04940a6aa89f418e88842dd3a13ed374&scope.project.id=3d22f09fa68241c08b79554641f15ddd&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:08.371543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb1f09-57c1-4586-b267-ca72e9bba6f9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:08.372086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb1f09-57c1-4586-b267-ca72e9bba6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:08.372086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb1f09-57c1-4586-b267-ca72e9bba6f9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:08.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 560/1107] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:08.379130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb514974-ccd7-4073-8495-4f33f389c704 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:08.379547 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb514974-ccd7-4073-8495-4f33f389c704 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:08.379681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb514974-ccd7-4073-8495-4f33f389c704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:08.396454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-fb514974-ccd7-4073-8495-4f33f389c704 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:08.397758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fb514974-ccd7-4073-8495-4f33f389c704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:08.460412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 548/1108] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:08.484328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb1f09-57c1-4586-b267-ca72e9bba6f9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:08.583635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb1f09-57c1-4586-b267-ca72e9bba6f9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:08.616058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be6466-8229-42cf-883b-14ce68dc203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:08.616867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be6466-8229-42cf-883b-14ce68dc203e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:08.617155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be6466-8229-42cf-883b-14ce68dc203e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:08.617432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be6466-8229-42cf-883b-14ce68dc203e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:08.618586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be6466-8229-42cf-883b-14ce68dc203e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:08.620010 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be6466-8229-42cf-883b-14ce68dc203e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:08.621665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 561/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:52:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:08.627978 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb514974-ccd7-4073-8495-4f33f389c704 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:08.718358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:08.721961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb514974-ccd7-4073-8495-4f33f389c704 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:08.754609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14046a-4ef2-4537-8a80-0e3722a2d1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:08.755253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14046a-4ef2-4537-8a80-0e3722a2d1d2 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:08.755620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14046a-4ef2-4537-8a80-0e3722a2d1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:08.756098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14046a-4ef2-4537-8a80-0e3722a2d1d2 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:08.757077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14046a-4ef2-4537-8a80-0e3722a2d1d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:08.758852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14046a-4ef2-4537-8a80-0e3722a2d1d2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:08.847747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 549/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:52:08 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:52:10.897397 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b40ed7-7e2b-4d77-99a4-40570e53d531 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:10.897962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b40ed7-7e2b-4d77-99a4-40570e53d531 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:10.897962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b40ed7-7e2b-4d77-99a4-40570e53d531 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:10.899302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 562/1111] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:10.904465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a968a-9a16-4405-aa04-8bb0361f6121 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:10.904899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a968a-9a16-4405-aa04-8bb0361f6121 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:10.905065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a968a-9a16-4405-aa04-8bb0361f6121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:10.921342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-870a968a-9a16-4405-aa04-8bb0361f6121 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:10.922538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-870a968a-9a16-4405-aa04-8bb0361f6121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:10.987046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 550/1112] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:11.010347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b40ed7-7e2b-4d77-99a4-40570e53d531 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:11.022385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:11.113811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b40ed7-7e2b-4d77-99a4-40570e53d531 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:11.146231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896948e-c632-4880-a59e-10c8c51d9edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:11.147064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896948e-c632-4880-a59e-10c8c51d9edf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:11.147346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896948e-c632-4880-a59e-10c8c51d9edf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:11.147667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896948e-c632-4880-a59e-10c8c51d9edf None admin] PATH_INFO: `/v3/roles/2991af796dcd45e2b86208339339b977` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:11.149098 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896948e-c632-4880-a59e-10c8c51d9edf None admin] RBAC: Authorizing `identity:get_role(role_id=2991af796dcd45e2b86208339339b977)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:11.150466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896948e-c632-4880-a59e-10c8c51d9edf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:11.151811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 563/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:11 2022] GET /identity/v3/roles/2991af796dcd45e2b86208339339b977 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:11.157797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a968a-9a16-4405-aa04-8bb0361f6121 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:11.237284 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:11.252487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a968a-9a16-4405-aa04-8bb0361f6121 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:11.284415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ced65-6b8c-416d-8b29-f773d1c9a6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:11.285290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ced65-6b8c-416d-8b29-f773d1c9a6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:11.285571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ced65-6b8c-416d-8b29-f773d1c9a6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:11.285845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ced65-6b8c-416d-8b29-f773d1c9a6d2 None admin] PATH_INFO: `/v3/users/7472a1c5399b4126a62847a9d78fffb7` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:11.299559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ced65-6b8c-416d-8b29-f773d1c9a6d2 None admin] RBAC: Authorizing `identity:get_user(user_id=7472a1c5399b4126a62847a9d78fffb7)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:11.301005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ced65-6b8c-416d-8b29-f773d1c9a6d2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:11.302634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 551/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:11 2022] GET /identity/v3/users/7472a1c5399b4126a62847a9d78fffb7 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:11.308250 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896948e-c632-4880-a59e-10c8c51d9edf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:11.385947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:11.407030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896948e-c632-4880-a59e-10c8c51d9edf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:11.439919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753bbac-9c29-4262-bb63-86e88338744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:11.440548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bbac-9c29-4262-bb63-86e88338744a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:11.440862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bbac-9c29-4262-bb63-86e88338744a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:11.441139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bbac-9c29-4262-bb63-86e88338744a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:11.505901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753bbac-9c29-4262-bb63-86e88338744a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:11.507346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753bbac-9c29-4262-bb63-86e88338744a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:11.579943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:11.601514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 564/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:52:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:52:11.607568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ced65-6b8c-416d-8b29-f773d1c9a6d2 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:11.702946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ced65-6b8c-416d-8b29-f773d1c9a6d2 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:11.735564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962ca31-9ddf-402b-877b-f222dd6afa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:11.736393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962ca31-9ddf-402b-877b-f222dd6afa4e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:11.736674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962ca31-9ddf-402b-877b-f222dd6afa4e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:11.736970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962ca31-9ddf-402b-877b-f222dd6afa4e None admin] PATH_INFO: `/v3/projects/3d22f09fa68241c08b79554641f15ddd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:11.738141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962ca31-9ddf-402b-877b-f222dd6afa4e None admin] RBAC: Authorizing `identity:get_project(project_id=3d22f09fa68241c08b79554641f15ddd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:11.739563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962ca31-9ddf-402b-877b-f222dd6afa4e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:11.741240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 552/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:11 2022] GET /identity/v3/projects/3d22f09fa68241c08b79554641f15ddd => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:11.746987 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753bbac-9c29-4262-bb63-86e88338744a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:11.842260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753bbac-9c29-4262-bb63-86e88338744a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:11.875299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15431a48-58f2-43be-b89a-99b7b7874df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:11.875932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15431a48-58f2-43be-b89a-99b7b7874df3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:11.876217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15431a48-58f2-43be-b89a-99b7b7874df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:11.876494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15431a48-58f2-43be-b89a-99b7b7874df3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:11.877353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15431a48-58f2-43be-b89a-99b7b7874df3 None admin] RBAC: Adding query filter params (role.id=2991af796dcd45e2b86208339339b977, scope.project.id=3d22f09fa68241c08b79554641f15ddd, user.id=7472a1c5399b4126a62847a9d78fffb7) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:11.877664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15431a48-58f2-43be-b89a-99b7b7874df3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:11.879077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15431a48-58f2-43be-b89a-99b7b7874df3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:11.884341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 565/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:52:11 2022] GET /identity/v3/role_assignments?user.id=7472a1c5399b4126a62847a9d78fffb7&scope.project.id=3d22f09fa68241c08b79554641f15ddd&role.id=2991af796dcd45e2b86208339339b977 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:13.861286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593622a1-2866-41c6-8e3a-85846bc3c458 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:13.861833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593622a1-2866-41c6-8e3a-85846bc3c458 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:13.861833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593622a1-2866-41c6-8e3a-85846bc3c458 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:13.863266 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 553/1118] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:13.868518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3604c-6f6e-4d33-80c5-4861a21efaa7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:13.868981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3604c-6f6e-4d33-80c5-4861a21efaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:13.869100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3604c-6f6e-4d33-80c5-4861a21efaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:13.885884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-00c3604c-6f6e-4d33-80c5-4861a21efaa7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:13.887061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-00c3604c-6f6e-4d33-80c5-4861a21efaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:13.923582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:13.955711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 566/1119] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:13.977954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593622a1-2866-41c6-8e3a-85846bc3c458 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:13.990475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:14.075947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593622a1-2866-41c6-8e3a-85846bc3c458 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:14.107751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2c123-7344-4742-8a28-c924cab5f2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:14.109097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c123-7344-4742-8a28-c924cab5f2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:14.109399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c123-7344-4742-8a28-c924cab5f2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:14.109668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c123-7344-4742-8a28-c924cab5f2a4 None admin] PATH_INFO: `/v3/roles/2991af796dcd45e2b86208339339b977` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:14.111106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2c123-7344-4742-8a28-c924cab5f2a4 None admin] RBAC: Authorizing `identity:get_role(role_id=2991af796dcd45e2b86208339339b977)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:14.112496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2c123-7344-4742-8a28-c924cab5f2a4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:14.113796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 554/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:13 2022] GET /identity/v3/roles/2991af796dcd45e2b86208339339b977 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:14.119490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3604c-6f6e-4d33-80c5-4861a21efaa7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:14.214601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3604c-6f6e-4d33-80c5-4861a21efaa7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:14.247218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc49f66-72d0-458b-a4a1-b6be5a79cadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:14.248061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49f66-72d0-458b-a4a1-b6be5a79cadd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:14.248345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49f66-72d0-458b-a4a1-b6be5a79cadd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:14.248620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc49f66-72d0-458b-a4a1-b6be5a79cadd None admin] PATH_INFO: `/v3/users/7472a1c5399b4126a62847a9d78fffb7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:14.250394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc49f66-72d0-458b-a4a1-b6be5a79cadd None admin] RBAC: Authorizing `identity:get_user(user_id=7472a1c5399b4126a62847a9d78fffb7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:14.251829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc49f66-72d0-458b-a4a1-b6be5a79cadd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:14.253438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 567/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:14 2022] GET /identity/v3/users/7472a1c5399b4126a62847a9d78fffb7 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:14.258989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2c123-7344-4742-8a28-c924cab5f2a4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:14.335363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:14.356022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2c123-7344-4742-8a28-c924cab5f2a4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:14.388707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38e5bb-400d-447e-b5c9-6308072c195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:14.389523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38e5bb-400d-447e-b5c9-6308072c195a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:14.389814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38e5bb-400d-447e-b5c9-6308072c195a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:14.390088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38e5bb-400d-447e-b5c9-6308072c195a None admin] PATH_INFO: `/v3/projects/3d22f09fa68241c08b79554641f15ddd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:14.391254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38e5bb-400d-447e-b5c9-6308072c195a None admin] RBAC: Authorizing `identity:get_project(project_id=3d22f09fa68241c08b79554641f15ddd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:14.392677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38e5bb-400d-447e-b5c9-6308072c195a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:14.394388 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 555/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:14 2022] GET /identity/v3/projects/3d22f09fa68241c08b79554641f15ddd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:14.400297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc49f66-72d0-458b-a4a1-b6be5a79cadd None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:14.441261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:14.497161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc49f66-72d0-458b-a4a1-b6be5a79cadd None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:14.529140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757aef1-baa9-4b13-a3d2-ba5dfe80801f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:14.530015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757aef1-baa9-4b13-a3d2-ba5dfe80801f None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:14.530300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757aef1-baa9-4b13-a3d2-ba5dfe80801f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:14.531088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757aef1-baa9-4b13-a3d2-ba5dfe80801f None admin] PATH_INFO: `/v3/projects/3d22f09fa68241c08b79554641f15ddd/users/7472a1c5399b4126a62847a9d78fffb7/roles/2991af796dcd45e2b86208339339b977` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:14.533259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757aef1-baa9-4b13-a3d2-ba5dfe80801f None admin] RBAC: Authorizing `identity:create_grant(project_id=3d22f09fa68241c08b79554641f15ddd, user_id=7472a1c5399b4126a62847a9d78fffb7, role_id=2991af796dcd45e2b86208339339b977)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:14.534679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757aef1-baa9-4b13-a3d2-ba5dfe80801f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:14.601498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 568/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:52:14 2022] PUT /identity/v3/projects/3d22f09fa68241c08b79554641f15ddd/users/7472a1c5399b4126a62847a9d78fffb7/roles/2991af796dcd45e2b86208339339b977 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:52:16.588409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e61fc-5200-40fc-a130-ff98a56a3882 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:16.588949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e61fc-5200-40fc-a130-ff98a56a3882 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:16.588949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e61fc-5200-40fc-a130-ff98a56a3882 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:16.590278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 556/1124] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:16.595859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d01a5-2b97-472e-980b-210c2063a7f6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:16.596241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d01a5-2b97-472e-980b-210c2063a7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:16.596414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d01a5-2b97-472e-980b-210c2063a7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:16.613612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ee8d01a5-2b97-472e-980b-210c2063a7f6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:16.614783 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ee8d01a5-2b97-472e-980b-210c2063a7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:16.678016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 569/1125] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:16.701026 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e61fc-5200-40fc-a130-ff98a56a3882 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:16.798130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e61fc-5200-40fc-a130-ff98a56a3882 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:16.830652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b67ced-e21f-489b-a313-3fd9fd0bac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:16.831484 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b67ced-e21f-489b-a313-3fd9fd0bac3f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:16.831771 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b67ced-e21f-489b-a313-3fd9fd0bac3f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:16.832046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b67ced-e21f-489b-a313-3fd9fd0bac3f None admin] PATH_INFO: `/v3/roles/2991af796dcd45e2b86208339339b977` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:16.833499 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b67ced-e21f-489b-a313-3fd9fd0bac3f None admin] RBAC: Authorizing `identity:get_role(role_id=2991af796dcd45e2b86208339339b977)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:16.834839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b67ced-e21f-489b-a313-3fd9fd0bac3f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:16.836153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 557/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:16 2022] GET /identity/v3/roles/2991af796dcd45e2b86208339339b977 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:16.841803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d01a5-2b97-472e-980b-210c2063a7f6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:16.870851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:16.937519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d01a5-2b97-472e-980b-210c2063a7f6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:16.970439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab36a03-5d3a-4d5f-9f95-a27c0c8d07fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:16.971283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab36a03-5d3a-4d5f-9f95-a27c0c8d07fc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:16.971562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab36a03-5d3a-4d5f-9f95-a27c0c8d07fc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:16.971830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab36a03-5d3a-4d5f-9f95-a27c0c8d07fc None admin] PATH_INFO: `/v3/users/7472a1c5399b4126a62847a9d78fffb7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:16.973073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab36a03-5d3a-4d5f-9f95-a27c0c8d07fc None admin] RBAC: Authorizing `identity:get_user(user_id=7472a1c5399b4126a62847a9d78fffb7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:16.974492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab36a03-5d3a-4d5f-9f95-a27c0c8d07fc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:16.976084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 570/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:16 2022] GET /identity/v3/users/7472a1c5399b4126a62847a9d78fffb7 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:16.981985 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b67ced-e21f-489b-a313-3fd9fd0bac3f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:16.997548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:17.078003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b67ced-e21f-489b-a313-3fd9fd0bac3f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:17.114921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02430a3-5540-4124-889d-38f01e67ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:17.115484 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02430a3-5540-4124-889d-38f01e67ecf9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:17.115762 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02430a3-5540-4124-889d-38f01e67ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:17.116031 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02430a3-5540-4124-889d-38f01e67ecf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:17.179778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02430a3-5540-4124-889d-38f01e67ecf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:17.181276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02430a3-5540-4124-889d-38f01e67ecf9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:17.195395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:17.280968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 558/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:52:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:52:17.286943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab36a03-5d3a-4d5f-9f95-a27c0c8d07fc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:17.381157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:17.384790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab36a03-5d3a-4d5f-9f95-a27c0c8d07fc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:17.417785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0abb3-c8ba-4422-800f-d0f7a9d9bf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:17.418621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0abb3-c8ba-4422-800f-d0f7a9d9bf63 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:17.418900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0abb3-c8ba-4422-800f-d0f7a9d9bf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:17.419169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0abb3-c8ba-4422-800f-d0f7a9d9bf63 None admin] PATH_INFO: `/v3/projects/3d22f09fa68241c08b79554641f15ddd` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:17.420326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0abb3-c8ba-4422-800f-d0f7a9d9bf63 None admin] RBAC: Authorizing `identity:get_project(project_id=3d22f09fa68241c08b79554641f15ddd)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:17.422451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0abb3-c8ba-4422-800f-d0f7a9d9bf63 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:17.424103 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 571/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:17 2022] GET /identity/v3/projects/3d22f09fa68241c08b79554641f15ddd => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:17.430154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02430a3-5540-4124-889d-38f01e67ecf9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:17.523934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02430a3-5540-4124-889d-38f01e67ecf9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:17.537207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:17.556401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8aaea2-9dfa-4d6e-ad7e-3a6e3df07745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:17.557068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aaea2-9dfa-4d6e-ad7e-3a6e3df07745 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:17.557347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aaea2-9dfa-4d6e-ad7e-3a6e3df07745 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:17.557618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aaea2-9dfa-4d6e-ad7e-3a6e3df07745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:17.558437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8aaea2-9dfa-4d6e-ad7e-3a6e3df07745 None admin] RBAC: Adding query filter params (role.id=2991af796dcd45e2b86208339339b977, scope.project.id=3d22f09fa68241c08b79554641f15ddd, user.id=7472a1c5399b4126a62847a9d78fffb7) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:17.558747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8aaea2-9dfa-4d6e-ad7e-3a6e3df07745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:17.560175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8aaea2-9dfa-4d6e-ad7e-3a6e3df07745 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:17.566147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 559/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:52:17 2022] GET /identity/v3/role_assignments?user.id=7472a1c5399b4126a62847a9d78fffb7&scope.project.id=3d22f09fa68241c08b79554641f15ddd&role.id=2991af796dcd45e2b86208339339b977 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:19.684299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecb629-bdfa-4e37-b81e-c7a1c8d6d4ad None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:19.684837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecb629-bdfa-4e37-b81e-c7a1c8d6d4ad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:19.685011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecb629-bdfa-4e37-b81e-c7a1c8d6d4ad None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:19.686394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 572/1131] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:19.692013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc82d2-1311-4d25-bf45-7e09bcaad36d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:19.692420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc82d2-1311-4d25-bf45-7e09bcaad36d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:19.692580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc82d2-1311-4d25-bf45-7e09bcaad36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:19.709581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3bdc82d2-1311-4d25-bf45-7e09bcaad36d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:19.710757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3bdc82d2-1311-4d25-bf45-7e09bcaad36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:19.775078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 560/1132] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:19.797937 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecb629-bdfa-4e37-b81e-c7a1c8d6d4ad None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:19.896126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecb629-bdfa-4e37-b81e-c7a1c8d6d4ad None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:19.927951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66622cfd-4ab9-43ab-81a1-04a6ee47278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:19.928712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66622cfd-4ab9-43ab-81a1-04a6ee47278a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:19.929048 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66622cfd-4ab9-43ab-81a1-04a6ee47278a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:19.929288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66622cfd-4ab9-43ab-81a1-04a6ee47278a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:19.930434 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66622cfd-4ab9-43ab-81a1-04a6ee47278a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:19.931845 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66622cfd-4ab9-43ab-81a1-04a6ee47278a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:19.933434 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 573/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:52:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:19.939824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc82d2-1311-4d25-bf45-7e09bcaad36d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:20.036011 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc82d2-1311-4d25-bf45-7e09bcaad36d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:20.044960 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:20.069433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199daa7-6135-4f70-8200-384408f2d391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:20.070080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199daa7-6135-4f70-8200-384408f2d391 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:20.070366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199daa7-6135-4f70-8200-384408f2d391 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:20.070644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199daa7-6135-4f70-8200-384408f2d391 None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:20.071357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199daa7-6135-4f70-8200-384408f2d391 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:20.072791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199daa7-6135-4f70-8200-384408f2d391 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:20.160402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 561/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:52:19 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:52:22.210797 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8ecef-888b-4b0e-8fba-79d91ba46c84 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:22.211349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8ecef-888b-4b0e-8fba-79d91ba46c84 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:22.211349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8ecef-888b-4b0e-8fba-79d91ba46c84 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:22.212705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 574/1135] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:22.217791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcc68a-2a6c-4099-939e-a96a2453d194 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:22.218160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcc68a-2a6c-4099-939e-a96a2453d194 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:22.218359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcc68a-2a6c-4099-939e-a96a2453d194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:22.235505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-cebcc68a-2a6c-4099-939e-a96a2453d194 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:22.236685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-cebcc68a-2a6c-4099-939e-a96a2453d194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:22.301239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 562/1136] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:22.326790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8ecef-888b-4b0e-8fba-79d91ba46c84 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:22.361716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:22.426711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8ecef-888b-4b0e-8fba-79d91ba46c84 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:22.459501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca35e0e-dc17-4946-8ba2-4a18ea8fd326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:22.460267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca35e0e-dc17-4946-8ba2-4a18ea8fd326 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:22.460547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca35e0e-dc17-4946-8ba2-4a18ea8fd326 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:22.460896 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca35e0e-dc17-4946-8ba2-4a18ea8fd326 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:22.462004 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca35e0e-dc17-4946-8ba2-4a18ea8fd326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:22.463418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca35e0e-dc17-4946-8ba2-4a18ea8fd326 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:22.465019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 575/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:52:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:22.470585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcc68a-2a6c-4099-939e-a96a2453d194 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:22.556991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:22.566839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcc68a-2a6c-4099-939e-a96a2453d194 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:22.599903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542a873-4d73-4038-b967-d13390017fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:22.600573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542a873-4d73-4038-b967-d13390017fd6 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:22.600888 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542a873-4d73-4038-b967-d13390017fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:22.601165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542a873-4d73-4038-b967-d13390017fd6 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:22.602136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542a873-4d73-4038-b967-d13390017fd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:22.603571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542a873-4d73-4038-b967-d13390017fd6 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:22.669711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 563/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:52:22 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:52:24.696805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2a790-18d7-472c-92e6-3a74f7944f9e None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:24.697357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2a790-18d7-472c-92e6-3a74f7944f9e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:24.697357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2a790-18d7-472c-92e6-3a74f7944f9e None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:24.698715 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 576/1139] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:24.703798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d3f15-0910-4089-b041-686482ed8a33 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:24.704223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d3f15-0910-4089-b041-686482ed8a33 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:24.704366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d3f15-0910-4089-b041-686482ed8a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:24.720732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-912d3f15-0910-4089-b041-686482ed8a33 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:24.721928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-912d3f15-0910-4089-b041-686482ed8a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:24.786741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 564/1140] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:24.809702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2a790-18d7-472c-92e6-3a74f7944f9e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:24.907019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2a790-18d7-472c-92e6-3a74f7944f9e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:24.908489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:24.939872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6ab51-2dda-4e92-8dda-37069b22f046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:24.941038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6ab51-2dda-4e92-8dda-37069b22f046 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:24.941330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6ab51-2dda-4e92-8dda-37069b22f046 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:24.941811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6ab51-2dda-4e92-8dda-37069b22f046 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:24.952382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6ab51-2dda-4e92-8dda-37069b22f046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-77b6ab51-2dda-4e92-8dda-37069b22f046 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:52:24.959840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:24.960882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:24.961861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 577/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:52:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:24.966921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d3f15-0910-4089-b041-686482ed8a33 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:25.040296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:25.063014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d3f15-0910-4089-b041-686482ed8a33 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:25.096126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de3673-f3bb-4d39-bace-f94e8bc75776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:25.096797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de3673-f3bb-4d39-bace-f94e8bc75776 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:25.097211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de3673-f3bb-4d39-bace-f94e8bc75776 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:25.097526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de3673-f3bb-4d39-bace-f94e8bc75776 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:25.098297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de3673-f3bb-4d39-bace-f94e8bc75776 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:25.098779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de3673-f3bb-4d39-bace-f94e8bc75776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:25.100880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de3673-f3bb-4d39-bace-f94e8bc75776 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:25.109641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 565/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:52:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:25.115144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6ab51-2dda-4e92-8dda-37069b22f046 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:25.216422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6ab51-2dda-4e92-8dda-37069b22f046 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:25.249001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23312c0f-b465-46ad-9988-6cf49b8b2cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:25.249857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23312c0f-b465-46ad-9988-6cf49b8b2cde None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:25.250144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23312c0f-b465-46ad-9988-6cf49b8b2cde None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:25.250425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23312c0f-b465-46ad-9988-6cf49b8b2cde None admin] PATH_INFO: `/v3/users/fc88a8bab9de4064bd2c4ff4c786ac74` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:25.263108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23312c0f-b465-46ad-9988-6cf49b8b2cde None admin] RBAC: Authorizing `identity:get_user(user_id=fc88a8bab9de4064bd2c4ff4c786ac74)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:25.265088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23312c0f-b465-46ad-9988-6cf49b8b2cde None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:25.266714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 578/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:25 2022] GET /identity/v3/users/fc88a8bab9de4064bd2c4ff4c786ac74 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:25.272513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de3673-f3bb-4d39-bace-f94e8bc75776 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:25.368501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de3673-f3bb-4d39-bace-f94e8bc75776 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:25.400242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c94863-79e3-4f13-aa05-43cbb1389cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:25.400889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c94863-79e3-4f13-aa05-43cbb1389cfa None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:25.401172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c94863-79e3-4f13-aa05-43cbb1389cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:25.401449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c94863-79e3-4f13-aa05-43cbb1389cfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:25.410062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:25.465205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c94863-79e3-4f13-aa05-43cbb1389cfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:25.466665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c94863-79e3-4f13-aa05-43cbb1389cfa None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:25.560309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 566/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:52:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:52:25.566140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23312c0f-b465-46ad-9988-6cf49b8b2cde None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:25.587557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:25.662963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23312c0f-b465-46ad-9988-6cf49b8b2cde None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:25.696335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b924c-7202-4c32-8c80-32e35a3ad4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:25.697204 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b924c-7202-4c32-8c80-32e35a3ad4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:25.697729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b924c-7202-4c32-8c80-32e35a3ad4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:25.697729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b924c-7202-4c32-8c80-32e35a3ad4a2 None admin] PATH_INFO: `/v3/projects/e154e10d5cce4890957b0b6d0b8f0f56` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:25.698894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b924c-7202-4c32-8c80-32e35a3ad4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=e154e10d5cce4890957b0b6d0b8f0f56)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:25.700320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b924c-7202-4c32-8c80-32e35a3ad4a2 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:25.702632 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 579/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:25 2022] GET /identity/v3/projects/e154e10d5cce4890957b0b6d0b8f0f56 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:25.708528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c94863-79e3-4f13-aa05-43cbb1389cfa None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:25.745953 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:25.802104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c94863-79e3-4f13-aa05-43cbb1389cfa None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:25.834998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd952acb-3bf3-48db-9fb0-7eee204be8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:25.835628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd952acb-3bf3-48db-9fb0-7eee204be8bd None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:25.835911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd952acb-3bf3-48db-9fb0-7eee204be8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:25.836184 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd952acb-3bf3-48db-9fb0-7eee204be8bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:25.837032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd952acb-3bf3-48db-9fb0-7eee204be8bd None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=e154e10d5cce4890957b0b6d0b8f0f56, user.id=fc88a8bab9de4064bd2c4ff4c786ac74) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:25.837341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd952acb-3bf3-48db-9fb0-7eee204be8bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:25.838738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd952acb-3bf3-48db-9fb0-7eee204be8bd None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:25.843951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 567/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:52:25 2022] GET /identity/v3/role_assignments?user.id=fc88a8bab9de4064bd2c4ff4c786ac74&scope.project.id=e154e10d5cce4890957b0b6d0b8f0f56&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:27.816385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f9c26-ca6f-4e62-94d0-8b4112e20caf None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:27.816987 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f9c26-ca6f-4e62-94d0-8b4112e20caf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:27.816987 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f9c26-ca6f-4e62-94d0-8b4112e20caf None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:27.818326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 580/1147] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:27.823421 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86224071-0cb4-4666-ac41-082e29726728 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:27.823850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86224071-0cb4-4666-ac41-082e29726728 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:27.823980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86224071-0cb4-4666-ac41-082e29726728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:27.840659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-86224071-0cb4-4666-ac41-082e29726728 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:27.842018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-86224071-0cb4-4666-ac41-082e29726728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:27.909355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 568/1148] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:27.930922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f9c26-ca6f-4e62-94d0-8b4112e20caf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:28.017023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:28.029145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f9c26-ca6f-4e62-94d0-8b4112e20caf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:28.061938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86c7cc-b28f-4701-8422-aadc7da31729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:28.062701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86c7cc-b28f-4701-8422-aadc7da31729 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:28.062984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86c7cc-b28f-4701-8422-aadc7da31729 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:28.063257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86c7cc-b28f-4701-8422-aadc7da31729 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:28.073128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86c7cc-b28f-4701-8422-aadc7da31729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ad86c7cc-b28f-4701-8422-aadc7da31729 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:52:28.081129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:28.082213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:28.083382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 581/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:28.088139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86224071-0cb4-4666-ac41-082e29726728 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:28.117709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:28.183837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86224071-0cb4-4666-ac41-082e29726728 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:28.216521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0821483-45f9-4a96-a441-7778cb6cbe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:28.217191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0821483-45f9-4a96-a441-7778cb6cbe1a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:28.217472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0821483-45f9-4a96-a441-7778cb6cbe1a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:28.217742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0821483-45f9-4a96-a441-7778cb6cbe1a None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:28.218469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0821483-45f9-4a96-a441-7778cb6cbe1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:28.218769 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0821483-45f9-4a96-a441-7778cb6cbe1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:28.220113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0821483-45f9-4a96-a441-7778cb6cbe1a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:28.228027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 569/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:28.234140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86c7cc-b28f-4701-8422-aadc7da31729 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:28.331494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86c7cc-b28f-4701-8422-aadc7da31729 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:28.364083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0dca5-542a-4402-8d36-54e6cd15c65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:28.364964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0dca5-542a-4402-8d36-54e6cd15c65d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:28.365249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0dca5-542a-4402-8d36-54e6cd15c65d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:28.365525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0dca5-542a-4402-8d36-54e6cd15c65d None admin] PATH_INFO: `/v3/users/fc88a8bab9de4064bd2c4ff4c786ac74` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:28.366686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0dca5-542a-4402-8d36-54e6cd15c65d None admin] RBAC: Authorizing `identity:get_user(user_id=fc88a8bab9de4064bd2c4ff4c786ac74)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:28.368112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0dca5-542a-4402-8d36-54e6cd15c65d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:28.369739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 582/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:28 2022] GET /identity/v3/users/fc88a8bab9de4064bd2c4ff4c786ac74 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:28.376043 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0821483-45f9-4a96-a441-7778cb6cbe1a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:28.465210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:28.471396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0821483-45f9-4a96-a441-7778cb6cbe1a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:28.504777 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d3887-706b-4a22-a825-1a03d59df688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:28.505647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d3887-706b-4a22-a825-1a03d59df688 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:28.505928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d3887-706b-4a22-a825-1a03d59df688 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:28.506202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d3887-706b-4a22-a825-1a03d59df688 None admin] PATH_INFO: `/v3/projects/e154e10d5cce4890957b0b6d0b8f0f56` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:28.507353 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d3887-706b-4a22-a825-1a03d59df688 None admin] RBAC: Authorizing `identity:get_project(project_id=e154e10d5cce4890957b0b6d0b8f0f56)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:28.508774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d3887-706b-4a22-a825-1a03d59df688 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:28.510450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 570/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:28 2022] GET /identity/v3/projects/e154e10d5cce4890957b0b6d0b8f0f56 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:28.516467 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0dca5-542a-4402-8d36-54e6cd15c65d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:28.534373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:28.614099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0dca5-542a-4402-8d36-54e6cd15c65d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:28.646125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dbcbc-549b-448f-9ae5-48bcaa2a4f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:28.646983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dbcbc-549b-448f-9ae5-48bcaa2a4f33 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:28.647271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dbcbc-549b-448f-9ae5-48bcaa2a4f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:28.647548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dbcbc-549b-448f-9ae5-48bcaa2a4f33 None admin] PATH_INFO: `/v3/projects/e154e10d5cce4890957b0b6d0b8f0f56/users/fc88a8bab9de4064bd2c4ff4c786ac74/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:28.649462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dbcbc-549b-448f-9ae5-48bcaa2a4f33 None admin] RBAC: Authorizing `identity:create_grant(project_id=e154e10d5cce4890957b0b6d0b8f0f56, user_id=fc88a8bab9de4064bd2c4ff4c786ac74, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:28.650879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dbcbc-549b-448f-9ae5-48bcaa2a4f33 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:28.755387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 583/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:52:28 2022] PUT /identity/v3/projects/e154e10d5cce4890957b0b6d0b8f0f56/users/fc88a8bab9de4064bd2c4ff4c786ac74/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:52:30.749722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9bb3f-0eb7-42f5-ae98-0a92c96a0299 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:30.750281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9bb3f-0eb7-42f5-ae98-0a92c96a0299 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:30.750281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9bb3f-0eb7-42f5-ae98-0a92c96a0299 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:30.751635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 571/1154] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:30.756979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760cf9f-d7e0-44d8-a910-5642111bef05 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:30.757378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760cf9f-d7e0-44d8-a910-5642111bef05 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:30.757537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760cf9f-d7e0-44d8-a910-5642111bef05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:30.774280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e760cf9f-d7e0-44d8-a910-5642111bef05 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:30.775443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e760cf9f-d7e0-44d8-a910-5642111bef05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:30.839129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 584/1155] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:30.862963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9bb3f-0eb7-42f5-ae98-0a92c96a0299 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:30.961779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9bb3f-0eb7-42f5-ae98-0a92c96a0299 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:31.000427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886e629-4573-498c-90c5-aa43f36bf6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:31.001154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886e629-4573-498c-90c5-aa43f36bf6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:31.001435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886e629-4573-498c-90c5-aa43f36bf6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:31.001707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886e629-4573-498c-90c5-aa43f36bf6f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:31.011446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0886e629-4573-498c-90c5-aa43f36bf6f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-0886e629-4573-498c-90c5-aa43f36bf6f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:52:31.019007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:31.020163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:52:31.021322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 572/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:52:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:31.026152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760cf9f-d7e0-44d8-a910-5642111bef05 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:31.034732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:31.122675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760cf9f-d7e0-44d8-a910-5642111bef05 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:31.154410 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f952e7b-edb6-4398-a349-3103f67a9310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:31.155560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952e7b-edb6-4398-a349-3103f67a9310 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:31.155839 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952e7b-edb6-4398-a349-3103f67a9310 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:31.156110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952e7b-edb6-4398-a349-3103f67a9310 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:31.156871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952e7b-edb6-4398-a349-3103f67a9310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:31.157174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952e7b-edb6-4398-a349-3103f67a9310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:31.158531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952e7b-edb6-4398-a349-3103f67a9310 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:31.166240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 585/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:52:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:31.172413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886e629-4573-498c-90c5-aa43f36bf6f9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:31.197237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:31.275434 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886e629-4573-498c-90c5-aa43f36bf6f9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:31.308916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159b925-3ed7-4cbc-bcbb-32df9f95f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:31.309786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159b925-3ed7-4cbc-bcbb-32df9f95f03e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:31.310068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159b925-3ed7-4cbc-bcbb-32df9f95f03e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:31.310345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159b925-3ed7-4cbc-bcbb-32df9f95f03e None admin] PATH_INFO: `/v3/users/fc88a8bab9de4064bd2c4ff4c786ac74` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:31.311515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159b925-3ed7-4cbc-bcbb-32df9f95f03e None admin] RBAC: Authorizing `identity:get_user(user_id=fc88a8bab9de4064bd2c4ff4c786ac74)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:31.313006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159b925-3ed7-4cbc-bcbb-32df9f95f03e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:31.314614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 573/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:31 2022] GET /identity/v3/users/fc88a8bab9de4064bd2c4ff4c786ac74 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:31.320350 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f952e7b-edb6-4398-a349-3103f67a9310 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:31.385639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:31.418957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f952e7b-edb6-4398-a349-3103f67a9310 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:31.450733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c5399-d3a9-4e2f-977a-f016afeeca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:31.451370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c5399-d3a9-4e2f-977a-f016afeeca56 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:31.452189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c5399-d3a9-4e2f-977a-f016afeeca56 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:31.452471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c5399-d3a9-4e2f-977a-f016afeeca56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:31.517388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c5399-d3a9-4e2f-977a-f016afeeca56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:31.518840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c5399-d3a9-4e2f-977a-f016afeeca56 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:31.580551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:31.614547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 586/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:52:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:52:31.620426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159b925-3ed7-4cbc-bcbb-32df9f95f03e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:31.707544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:31.716325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159b925-3ed7-4cbc-bcbb-32df9f95f03e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:31.749487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b430a7-9e09-42a0-830a-e5b30475f005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:31.750328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b430a7-9e09-42a0-830a-e5b30475f005 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:31.750850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b430a7-9e09-42a0-830a-e5b30475f005 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:31.750850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b430a7-9e09-42a0-830a-e5b30475f005 None admin] PATH_INFO: `/v3/projects/e154e10d5cce4890957b0b6d0b8f0f56` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:31.752025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b430a7-9e09-42a0-830a-e5b30475f005 None admin] RBAC: Authorizing `identity:get_project(project_id=e154e10d5cce4890957b0b6d0b8f0f56)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:31.753476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b430a7-9e09-42a0-830a-e5b30475f005 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:31.755134 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 574/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:31 2022] GET /identity/v3/projects/e154e10d5cce4890957b0b6d0b8f0f56 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:31.761041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c5399-d3a9-4e2f-977a-f016afeeca56 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:31.857139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c5399-d3a9-4e2f-977a-f016afeeca56 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:31.890439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39084e-f6e6-4c54-94fc-fc92c9baab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:31.891077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39084e-f6e6-4c54-94fc-fc92c9baab74 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:31.891365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39084e-f6e6-4c54-94fc-fc92c9baab74 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:31.891649 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39084e-f6e6-4c54-94fc-fc92c9baab74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:31.892477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39084e-f6e6-4c54-94fc-fc92c9baab74 None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=e154e10d5cce4890957b0b6d0b8f0f56, user.id=fc88a8bab9de4064bd2c4ff4c786ac74) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:31.892814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39084e-f6e6-4c54-94fc-fc92c9baab74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:31.894252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39084e-f6e6-4c54-94fc-fc92c9baab74 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:31.899897 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 587/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:52:31 2022] GET /identity/v3/role_assignments?user.id=fc88a8bab9de4064bd2c4ff4c786ac74&scope.project.id=e154e10d5cce4890957b0b6d0b8f0f56&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:33.927134 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95c806-a654-457f-b2bf-636d796f6270 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:33.927661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95c806-a654-457f-b2bf-636d796f6270 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:33.927752 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95c806-a654-457f-b2bf-636d796f6270 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:33.929070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 575/1162] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:33.934058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422e236-5a5c-404a-b07f-3eee323bf984 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:33.934444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422e236-5a5c-404a-b07f-3eee323bf984 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:33.934609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422e236-5a5c-404a-b07f-3eee323bf984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:33.952652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6422e236-5a5c-404a-b07f-3eee323bf984 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:33.953837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6422e236-5a5c-404a-b07f-3eee323bf984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:33.978309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:34.016661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 588/1163] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:34.038851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95c806-a654-457f-b2bf-636d796f6270 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:34.145046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95c806-a654-457f-b2bf-636d796f6270 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:34.177417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f83429-592c-4447-9101-115ac6d9e059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:34.177994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f83429-592c-4447-9101-115ac6d9e059 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:34.178278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f83429-592c-4447-9101-115ac6d9e059 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:34.178553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f83429-592c-4447-9101-115ac6d9e059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:34.205156 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:34.244627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f83429-592c-4447-9101-115ac6d9e059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:34.246098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f83429-592c-4447-9101-115ac6d9e059 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:34.340394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 576/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:52:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:52:34.346210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422e236-5a5c-404a-b07f-3eee323bf984 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:34.447165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422e236-5a5c-404a-b07f-3eee323bf984 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:34.479387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155125fe-9866-44e0-a0f0-85bc06202625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:34.480168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155125fe-9866-44e0-a0f0-85bc06202625 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:34.480447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155125fe-9866-44e0-a0f0-85bc06202625 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:34.480716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155125fe-9866-44e0-a0f0-85bc06202625 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:34.487747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155125fe-9866-44e0-a0f0-85bc06202625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:34.489215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155125fe-9866-44e0-a0f0-85bc06202625 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-155125fe-9866-44e0-a0f0-85bc06202625 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:34.503463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:34.504664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:34.505830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 589/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Tue Jul 5 15:52:34 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:34.510545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f83429-592c-4447-9101-115ac6d9e059 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:34.566588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:34.607195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f83429-592c-4447-9101-115ac6d9e059 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:34.639573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86222d97-f17f-4e47-8795-025d66986c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:34.640221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86222d97-f17f-4e47-8795-025d66986c06 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:34.640506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86222d97-f17f-4e47-8795-025d66986c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:34.640804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86222d97-f17f-4e47-8795-025d66986c06 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:34.641534 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86222d97-f17f-4e47-8795-025d66986c06 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:34.641844 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86222d97-f17f-4e47-8795-025d66986c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:34.643197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86222d97-f17f-4e47-8795-025d66986c06 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:34.650159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 577/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Tue Jul 5 15:52:34 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:36.643553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33517838-ad1b-4c3a-b935-3e1497187dd4 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:36.644067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33517838-ad1b-4c3a-b935-3e1497187dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:36.644067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33517838-ad1b-4c3a-b935-3e1497187dd4 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:36.645418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 590/1167] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:36.650534 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ad17-8c78-4540-95f7-92af7fe191a7 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:36.650951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ad17-8c78-4540-95f7-92af7fe191a7 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:36.651095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648ad17-8c78-4540-95f7-92af7fe191a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:36.667491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-7648ad17-8c78-4540-95f7-92af7fe191a7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:36.668683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7648ad17-8c78-4540-95f7-92af7fe191a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:36.734146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 578/1168] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:36.756536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33517838-ad1b-4c3a-b935-3e1497187dd4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:36.780467 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:36.855548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33517838-ad1b-4c3a-b935-3e1497187dd4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:36.888209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7c1ff-a4eb-42c6-a685-0f5b0fb2d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:36.888907 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7c1ff-a4eb-42c6-a685-0f5b0fb2d37d None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:36.889194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7c1ff-a4eb-42c6-a685-0f5b0fb2d37d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:36.889471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7c1ff-a4eb-42c6-a685-0f5b0fb2d37d None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:36.890206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7c1ff-a4eb-42c6-a685-0f5b0fb2d37d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:36.891594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7c1ff-a4eb-42c6-a685-0f5b0fb2d37d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:36.938969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 591/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Tue Jul 5 15:52:36 2022] POST /identity/v3/domains => generated 267 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:52:38.985809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e917e2-c2f7-42af-a744-1a6c369c9d9c None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:38.986389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e917e2-c2f7-42af-a744-1a6c369c9d9c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:38.986389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e917e2-c2f7-42af-a744-1a6c369c9d9c None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:38.987727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 579/1170] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:38.992969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668845db-1561-4827-ac07-b1a4c74521d4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:38.993398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668845db-1561-4827-ac07-b1a4c74521d4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:38.993526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668845db-1561-4827-ac07-b1a4c74521d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:39.009755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-668845db-1561-4827-ac07-b1a4c74521d4 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:39.010941 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-668845db-1561-4827-ac07-b1a4c74521d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:39.075365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 592/1171] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:39.099272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e917e2-c2f7-42af-a744-1a6c369c9d9c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:39.152197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:39.201118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e917e2-c2f7-42af-a744-1a6c369c9d9c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:39.234161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614c9f2-a4fa-44d3-8968-ec53e6d4dc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:39.235012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614c9f2-a4fa-44d3-8968-ec53e6d4dc43 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:39.235299 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614c9f2-a4fa-44d3-8968-ec53e6d4dc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:39.235581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614c9f2-a4fa-44d3-8968-ec53e6d4dc43 None admin] PATH_INFO: `/v3/domains/a42277adc18047ccbd2b6b2d00f131fa` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:39.246210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614c9f2-a4fa-44d3-8968-ec53e6d4dc43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a42277adc18047ccbd2b6b2d00f131fa)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:39.247615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614c9f2-a4fa-44d3-8968-ec53e6d4dc43 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:39.249800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 580/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Tue Jul 5 15:52:39 2022] GET /identity/v3/domains/a42277adc18047ccbd2b6b2d00f131fa => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:39.255733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668845db-1561-4827-ac07-b1a4c74521d4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:39.270101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:39.353376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668845db-1561-4827-ac07-b1a4c74521d4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:39.385596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315428d-7d8b-4806-af98-b51b47808a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:39.386240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315428d-7d8b-4806-af98-b51b47808a9f None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:39.386519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315428d-7d8b-4806-af98-b51b47808a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:39.386790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315428d-7d8b-4806-af98-b51b47808a9f None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:39.387497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315428d-7d8b-4806-af98-b51b47808a9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:39.389704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315428d-7d8b-4806-af98-b51b47808a9f None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:39.464419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 593/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:52:39 2022] POST /identity/v3/projects => generated 361 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:52:41.474994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93311f43-b04f-4f32-86ec-b3b2902b3217 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:41.475545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93311f43-b04f-4f32-86ec-b3b2902b3217 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:41.475545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93311f43-b04f-4f32-86ec-b3b2902b3217 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:41.477209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 581/1174] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:41.482563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634fcc9-a21d-4735-be10-8b9d2766396f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:41.482998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634fcc9-a21d-4735-be10-8b9d2766396f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:41.483106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634fcc9-a21d-4735-be10-8b9d2766396f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:41.499811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4634fcc9-a21d-4735-be10-8b9d2766396f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:41.500996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4634fcc9-a21d-4735-be10-8b9d2766396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:41.557808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:41.566160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 594/1175] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:41.590439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93311f43-b04f-4f32-86ec-b3b2902b3217 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:41.687897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93311f43-b04f-4f32-86ec-b3b2902b3217 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:41.700254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:41.720298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4796d27-3ffd-40a3-b657-0da1d8f8b886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:41.721163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4796d27-3ffd-40a3-b657-0da1d8f8b886 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:41.721446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4796d27-3ffd-40a3-b657-0da1d8f8b886 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:41.721719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4796d27-3ffd-40a3-b657-0da1d8f8b886 None admin] PATH_INFO: `/v3/domains/a42277adc18047ccbd2b6b2d00f131fa` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:41.722866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4796d27-3ffd-40a3-b657-0da1d8f8b886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a42277adc18047ccbd2b6b2d00f131fa)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:41.724275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4796d27-3ffd-40a3-b657-0da1d8f8b886 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:41.726234 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 582/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Tue Jul 5 15:52:41 2022] GET /identity/v3/domains/a42277adc18047ccbd2b6b2d00f131fa => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:41.731635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634fcc9-a21d-4735-be10-8b9d2766396f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:41.829032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634fcc9-a21d-4735-be10-8b9d2766396f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:41.862029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cb125-30c1-4dbc-9598-22e6443750ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:41.862682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cb125-30c1-4dbc-9598-22e6443750ca None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:41.862965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cb125-30c1-4dbc-9598-22e6443750ca None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:41.863239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cb125-30c1-4dbc-9598-22e6443750ca None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:41.863952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cb125-30c1-4dbc-9598-22e6443750ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:41.865403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cb125-30c1-4dbc-9598-22e6443750ca None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:41.960994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 595/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:52:41 2022] POST /identity/v3/users => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:52:43.997111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e0728-5916-4bbe-9286-fbf3a764b431 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:43.997663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e0728-5916-4bbe-9286-fbf3a764b431 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:43.997663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e0728-5916-4bbe-9286-fbf3a764b431 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:43.999592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 583/1178] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:44.004917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bc72e-bb67-4820-9d7e-154d0d2243d4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:44.005304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bc72e-bb67-4820-9d7e-154d0d2243d4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:44.005479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bc72e-bb67-4820-9d7e-154d0d2243d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:44.021823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b17bc72e-bb67-4820-9d7e-154d0d2243d4 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:44.023000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b17bc72e-bb67-4820-9d7e-154d0d2243d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:44.067817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:44.087479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 596/1179] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:44.111562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e0728-5916-4bbe-9286-fbf3a764b431 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:44.210269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e0728-5916-4bbe-9286-fbf3a764b431 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:44.243235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759937a-9ba6-465c-893b-d3e930bd2d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:44.244006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759937a-9ba6-465c-893b-d3e930bd2d74 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:44.244292 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759937a-9ba6-465c-893b-d3e930bd2d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:44.244569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759937a-9ba6-465c-893b-d3e930bd2d74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:44.255050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759937a-9ba6-465c-893b-d3e930bd2d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-b759937a-9ba6-465c-893b-d3e930bd2d74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:52:44.262603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:44.263792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:52:44.265042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 584/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:52:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:44.269620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bc72e-bb67-4820-9d7e-154d0d2243d4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:44.366671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bc72e-bb67-4820-9d7e-154d0d2243d4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:44.398718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f2750-26fd-4761-8568-d1b5b8a09710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:44.399373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2750-26fd-4761-8568-d1b5b8a09710 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:44.399658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2750-26fd-4761-8568-d1b5b8a09710 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:44.399934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2750-26fd-4761-8568-d1b5b8a09710 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:44.400679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f2750-26fd-4761-8568-d1b5b8a09710 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:44.401018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f2750-26fd-4761-8568-d1b5b8a09710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:44.402390 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f2750-26fd-4761-8568-d1b5b8a09710 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:44.411791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 597/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:52:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:44.417521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759937a-9ba6-465c-893b-d3e930bd2d74 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:44.458532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:44.515787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759937a-9ba6-465c-893b-d3e930bd2d74 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:44.548938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458a785-09b3-429b-9b09-c9719be37d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:44.549785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a785-09b3-429b-9b09-c9719be37d9f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:44.550071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a785-09b3-429b-9b09-c9719be37d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:44.550345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a785-09b3-429b-9b09-c9719be37d9f None admin] PATH_INFO: `/v3/users/d6b8ba4a4a0e49ae953fce0422beec74` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:44.563091 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458a785-09b3-429b-9b09-c9719be37d9f None admin] RBAC: Authorizing `identity:get_user(user_id=d6b8ba4a4a0e49ae953fce0422beec74)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:44.564558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458a785-09b3-429b-9b09-c9719be37d9f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:44.566207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 585/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:44 2022] GET /identity/v3/users/d6b8ba4a4a0e49ae953fce0422beec74 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:44.571843 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f2750-26fd-4761-8568-d1b5b8a09710 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:44.614915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:44.670646 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f2750-26fd-4761-8568-d1b5b8a09710 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:44.704717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc35c84-cf1d-4e65-abe7-247c76f84eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:44.705398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35c84-cf1d-4e65-abe7-247c76f84eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:44.705690 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35c84-cf1d-4e65-abe7-247c76f84eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:44.705969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35c84-cf1d-4e65-abe7-247c76f84eb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:44.773578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc35c84-cf1d-4e65-abe7-247c76f84eb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:44.774970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc35c84-cf1d-4e65-abe7-247c76f84eb8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:44.814748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:44.870558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 598/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:52:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:52:44.876388 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458a785-09b3-429b-9b09-c9719be37d9f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:44.970371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:44.974107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458a785-09b3-429b-9b09-c9719be37d9f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:45.006253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1c985-868e-41a4-a769-c43db756f434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:45.007099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1c985-868e-41a4-a769-c43db756f434 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:45.007381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1c985-868e-41a4-a769-c43db756f434 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:45.007654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1c985-868e-41a4-a769-c43db756f434 None admin] PATH_INFO: `/v3/projects/4aa95d04ea14425aae6a289b6507cc57` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:45.008837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1c985-868e-41a4-a769-c43db756f434 None admin] RBAC: Authorizing `identity:get_project(project_id=4aa95d04ea14425aae6a289b6507cc57)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:45.010264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1c985-868e-41a4-a769-c43db756f434 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:45.012044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 586/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:44 2022] GET /identity/v3/projects/4aa95d04ea14425aae6a289b6507cc57 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:45.018563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc35c84-cf1d-4e65-abe7-247c76f84eb8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:45.119614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc35c84-cf1d-4e65-abe7-247c76f84eb8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:45.152432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48659c1-11be-4111-854f-3d79210839d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:45.153095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48659c1-11be-4111-854f-3d79210839d4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:45.153374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48659c1-11be-4111-854f-3d79210839d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:45.153642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48659c1-11be-4111-854f-3d79210839d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:45.154485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48659c1-11be-4111-854f-3d79210839d4 None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=4aa95d04ea14425aae6a289b6507cc57, user.id=d6b8ba4a4a0e49ae953fce0422beec74) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:45.154793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48659c1-11be-4111-854f-3d79210839d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:45.156212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48659c1-11be-4111-854f-3d79210839d4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:45.161457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 599/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:52:45 2022] GET /identity/v3/role_assignments?user.id=d6b8ba4a4a0e49ae953fce0422beec74&scope.project.id=4aa95d04ea14425aae6a289b6507cc57&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:47.168554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c2ec6-db87-4575-8962-91514936bc2c None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:47.169135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c2ec6-db87-4575-8962-91514936bc2c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:47.169135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c2ec6-db87-4575-8962-91514936bc2c None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:47.171031 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 587/1186] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:47.176221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28b1f7-3e1e-4983-9f16-1e7f0703f0a8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:47.176604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28b1f7-3e1e-4983-9f16-1e7f0703f0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:47.176785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28b1f7-3e1e-4983-9f16-1e7f0703f0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:47.188822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:47.194056 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3d28b1f7-3e1e-4983-9f16-1e7f0703f0a8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:47.195240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3d28b1f7-3e1e-4983-9f16-1e7f0703f0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:47.263675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 600/1187] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:47.286128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c2ec6-db87-4575-8962-91514936bc2c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:47.386833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c2ec6-db87-4575-8962-91514936bc2c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:47.420566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b9ef4-5670-4e42-b4ea-a1b8ad287abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:47.421318 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b9ef4-5670-4e42-b4ea-a1b8ad287abb None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:47.421604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b9ef4-5670-4e42-b4ea-a1b8ad287abb None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:47.421880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b9ef4-5670-4e42-b4ea-a1b8ad287abb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:47.431704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b9ef4-5670-4e42-b4ea-a1b8ad287abb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-005b9ef4-5670-4e42-b4ea-a1b8ad287abb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:52:47.439191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:47.440270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:52:47.441176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 588/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:52:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:47.446666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28b1f7-3e1e-4983-9f16-1e7f0703f0a8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:47.542931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28b1f7-3e1e-4983-9f16-1e7f0703f0a8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:47.575785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de577d2-1918-45eb-b984-e5c41d0cabc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:47.576437 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de577d2-1918-45eb-b984-e5c41d0cabc8 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:47.576721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de577d2-1918-45eb-b984-e5c41d0cabc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:47.577030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de577d2-1918-45eb-b984-e5c41d0cabc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:47.577775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de577d2-1918-45eb-b984-e5c41d0cabc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:47.578086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de577d2-1918-45eb-b984-e5c41d0cabc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:47.579467 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de577d2-1918-45eb-b984-e5c41d0cabc8 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:47.581626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:47.587769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 601/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:52:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:47.594108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b9ef4-5670-4e42-b4ea-a1b8ad287abb None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:47.622122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:47.692157 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b9ef4-5670-4e42-b4ea-a1b8ad287abb None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:47.725039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d633aec-ef02-49a7-bf5f-3b3b06c327e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:47.725880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d633aec-ef02-49a7-bf5f-3b3b06c327e4 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:47.726164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d633aec-ef02-49a7-bf5f-3b3b06c327e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:47.726440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d633aec-ef02-49a7-bf5f-3b3b06c327e4 None admin] PATH_INFO: `/v3/users/d6b8ba4a4a0e49ae953fce0422beec74` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:47.727885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d633aec-ef02-49a7-bf5f-3b3b06c327e4 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b8ba4a4a0e49ae953fce0422beec74)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:47.729343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d633aec-ef02-49a7-bf5f-3b3b06c327e4 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:47.730929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 589/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:47 2022] GET /identity/v3/users/d6b8ba4a4a0e49ae953fce0422beec74 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:47.736659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de577d2-1918-45eb-b984-e5c41d0cabc8 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:47.831996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de577d2-1918-45eb-b984-e5c41d0cabc8 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:47.864585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166ddcc-db51-41ef-80c5-c15655ba29fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:47.865441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166ddcc-db51-41ef-80c5-c15655ba29fd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:47.865757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166ddcc-db51-41ef-80c5-c15655ba29fd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:47.866053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166ddcc-db51-41ef-80c5-c15655ba29fd None admin] PATH_INFO: `/v3/projects/4aa95d04ea14425aae6a289b6507cc57` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:47.867212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166ddcc-db51-41ef-80c5-c15655ba29fd None admin] RBAC: Authorizing `identity:get_project(project_id=4aa95d04ea14425aae6a289b6507cc57)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:47.868623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166ddcc-db51-41ef-80c5-c15655ba29fd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:47.870305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 602/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:47 2022] GET /identity/v3/projects/4aa95d04ea14425aae6a289b6507cc57 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:47.876467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d633aec-ef02-49a7-bf5f-3b3b06c327e4 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:47.965986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:47.972742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d633aec-ef02-49a7-bf5f-3b3b06c327e4 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:48.004906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e92d2-5f65-46ab-beab-8d089a79b17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:48.005765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e92d2-5f65-46ab-beab-8d089a79b17d None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:48.006045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e92d2-5f65-46ab-beab-8d089a79b17d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:48.006319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e92d2-5f65-46ab-beab-8d089a79b17d None admin] PATH_INFO: `/v3/projects/4aa95d04ea14425aae6a289b6507cc57/users/d6b8ba4a4a0e49ae953fce0422beec74/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:48.008225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e92d2-5f65-46ab-beab-8d089a79b17d None admin] RBAC: Authorizing `identity:create_grant(project_id=4aa95d04ea14425aae6a289b6507cc57, user_id=d6b8ba4a4a0e49ae953fce0422beec74, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:48.009649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e92d2-5f65-46ab-beab-8d089a79b17d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:48.078636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 590/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:52:47 2022] PUT /identity/v3/projects/4aa95d04ea14425aae6a289b6507cc57/users/d6b8ba4a4a0e49ae953fce0422beec74/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:52:50.105937 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd988e-5073-446c-93b2-6182e830fdb3 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:50.106491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd988e-5073-446c-93b2-6182e830fdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:50.106491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd988e-5073-446c-93b2-6182e830fdb3 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:50.107856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 603/1193] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:52:50.112917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba73d92-e590-4c1d-be85-ca87cacda973 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:50.113277 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba73d92-e590-4c1d-be85-ca87cacda973 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:50.113477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba73d92-e590-4c1d-be85-ca87cacda973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:50.129707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-dba73d92-e590-4c1d-be85-ca87cacda973 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:52:50.130883 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-dba73d92-e590-4c1d-be85-ca87cacda973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:52:50.195026 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 591/1194] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:52:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:52:50.219130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd988e-5073-446c-93b2-6182e830fdb3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:50.283946 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:50.319114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd988e-5073-446c-93b2-6182e830fdb3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:50.353297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ea106-7988-4bc4-ad2f-38f88fe9749c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:50.354077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ea106-7988-4bc4-ad2f-38f88fe9749c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:50.354359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ea106-7988-4bc4-ad2f-38f88fe9749c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:50.354633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ea106-7988-4bc4-ad2f-38f88fe9749c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:50.364478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ea106-7988-4bc4-ad2f-38f88fe9749c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-915ea106-7988-4bc4-ad2f-38f88fe9749c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:52:50.371855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:52:50.373503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:52:50.374522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 604/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:52:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:52:50.379580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba73d92-e590-4c1d-be85-ca87cacda973 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:50.459468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:50.474400 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba73d92-e590-4c1d-be85-ca87cacda973 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:50.507187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab852297-fc2f-48fc-9be2-db0186a575d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:50.507831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852297-fc2f-48fc-9be2-db0186a575d7 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:50.508114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852297-fc2f-48fc-9be2-db0186a575d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:50.508394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab852297-fc2f-48fc-9be2-db0186a575d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:50.509165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab852297-fc2f-48fc-9be2-db0186a575d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:50.509469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab852297-fc2f-48fc-9be2-db0186a575d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:50.510818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab852297-fc2f-48fc-9be2-db0186a575d7 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:50.519691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 592/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:52:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:50.525266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ea106-7988-4bc4-ad2f-38f88fe9749c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:50.621063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ea106-7988-4bc4-ad2f-38f88fe9749c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:50.646650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:50.654885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892489b0-dfee-457b-a629-27e3b4cbbed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:50.655728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892489b0-dfee-457b-a629-27e3b4cbbed4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:50.656013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892489b0-dfee-457b-a629-27e3b4cbbed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:50.656289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892489b0-dfee-457b-a629-27e3b4cbbed4 None admin] PATH_INFO: `/v3/users/d6b8ba4a4a0e49ae953fce0422beec74` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:50.657431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892489b0-dfee-457b-a629-27e3b4cbbed4 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b8ba4a4a0e49ae953fce0422beec74)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:50.658858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892489b0-dfee-457b-a629-27e3b4cbbed4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:50.660451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 605/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Tue Jul 5 15:52:50 2022] GET /identity/v3/users/d6b8ba4a4a0e49ae953fce0422beec74 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:50.665936 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab852297-fc2f-48fc-9be2-db0186a575d7 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:50.762239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab852297-fc2f-48fc-9be2-db0186a575d7 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:50.793631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd8c08-8fcf-42b0-98e3-6d60e4c35f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:50.794261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd8c08-8fcf-42b0-98e3-6d60e4c35f3c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:50.795080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd8c08-8fcf-42b0-98e3-6d60e4c35f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:50.795355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd8c08-8fcf-42b0-98e3-6d60e4c35f3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:50.809691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:50.860829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd8c08-8fcf-42b0-98e3-6d60e4c35f3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:50.862279 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd8c08-8fcf-42b0-98e3-6d60e4c35f3c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:50.955879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 593/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:52:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:52:50.961685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892489b0-dfee-457b-a629-27e3b4cbbed4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:51.057567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892489b0-dfee-457b-a629-27e3b4cbbed4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:51.089635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d88e17-7135-43eb-83c2-c1f31eedbedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:51.090465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d88e17-7135-43eb-83c2-c1f31eedbedc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:51.090747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d88e17-7135-43eb-83c2-c1f31eedbedc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:51.091024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d88e17-7135-43eb-83c2-c1f31eedbedc None admin] PATH_INFO: `/v3/projects/4aa95d04ea14425aae6a289b6507cc57` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:51.092713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d88e17-7135-43eb-83c2-c1f31eedbedc None admin] RBAC: Authorizing `identity:get_project(project_id=4aa95d04ea14425aae6a289b6507cc57)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:51.094161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d88e17-7135-43eb-83c2-c1f31eedbedc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:51.095804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 606/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Tue Jul 5 15:52:50 2022] GET /identity/v3/projects/4aa95d04ea14425aae6a289b6507cc57 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:52:51.101896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd8c08-8fcf-42b0-98e3-6d60e4c35f3c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:52:51.146169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:52:51.198171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd8c08-8fcf-42b0-98e3-6d60e4c35f3c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:52:51.230561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd64c0-a5c9-4c01-b41b-2338047116ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:52:51.231144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd64c0-a5c9-4c01-b41b-2338047116ee None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:52:51.231426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd64c0-a5c9-4c01-b41b-2338047116ee None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:52:51.231701 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd64c0-a5c9-4c01-b41b-2338047116ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:52:51.233073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd64c0-a5c9-4c01-b41b-2338047116ee None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=4aa95d04ea14425aae6a289b6507cc57, user.id=d6b8ba4a4a0e49ae953fce0422beec74) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:52:51.233384 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd64c0-a5c9-4c01-b41b-2338047116ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:52:51.234798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd64c0-a5c9-4c01-b41b-2338047116ee None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:52:51.240423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 594/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:52:51 2022] GET /identity/v3/role_assignments?user.id=d6b8ba4a4a0e49ae953fce0422beec74&scope.project.id=4aa95d04ea14425aae6a289b6507cc57&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:53:01.085673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=69 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:53:01.237890 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=45 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:03.854502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6be70d-a6dc-46e5-9d8a-77249cf5ad49 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:03.855089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6be70d-a6dc-46e5-9d8a-77249cf5ad49 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:03.855089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6be70d-a6dc-46e5-9d8a-77249cf5ad49 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:03.856408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 607/1201] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:03.861708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8542825-aca7-4278-b2ce-adac0156b2e6 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:03.862136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8542825-aca7-4278-b2ce-adac0156b2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:03.862308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8542825-aca7-4278-b2ce-adac0156b2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:03.879515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f8542825-aca7-4278-b2ce-adac0156b2e6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:03.880679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f8542825-aca7-4278-b2ce-adac0156b2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:03.945719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 595/1202] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:03.970474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6be70d-a6dc-46e5-9d8a-77249cf5ad49 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:04.070931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6be70d-a6dc-46e5-9d8a-77249cf5ad49 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:04.104934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce9d66-75c1-4b56-89fb-69b208034491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:04.105656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce9d66-75c1-4b56-89fb-69b208034491 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:04.105938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce9d66-75c1-4b56-89fb-69b208034491 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:04.106209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce9d66-75c1-4b56-89fb-69b208034491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:04.116489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce9d66-75c1-4b56-89fb-69b208034491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:04.117947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce9d66-75c1-4b56-89fb-69b208034491 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-61ce9d66-75c1-4b56-89fb-69b208034491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:04.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:04.139013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:04.140038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:04.140038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:04.140038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:04.140038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:04.140038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 608/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:04.145495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8542825-aca7-4278-b2ce-adac0156b2e6 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:04.242027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8542825-aca7-4278-b2ce-adac0156b2e6 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:04.271229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:04.276636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6538c7-6d88-4e68-95b4-8efbcc459f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:04.277317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6538c7-6d88-4e68-95b4-8efbcc459f67 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:04.277599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6538c7-6d88-4e68-95b4-8efbcc459f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:04.277863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6538c7-6d88-4e68-95b4-8efbcc459f67 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:04.278586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6538c7-6d88-4e68-95b4-8efbcc459f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:04.278889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6538c7-6d88-4e68-95b4-8efbcc459f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:04.280851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6538c7-6d88-4e68-95b4-8efbcc459f67 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:04.291956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 596/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:04.297581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce9d66-75c1-4b56-89fb-69b208034491 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:04.349145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:04.393695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce9d66-75c1-4b56-89fb-69b208034491 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:04.425683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e3efa-fce9-437a-b3c9-5b9699e87a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:04.426336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e3efa-fce9-437a-b3c9-5b9699e87a63 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:04.426619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e3efa-fce9-437a-b3c9-5b9699e87a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:04.426890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e3efa-fce9-437a-b3c9-5b9699e87a63 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:04.427588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e3efa-fce9-437a-b3c9-5b9699e87a63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:04.429062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e3efa-fce9-437a-b3c9-5b9699e87a63 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:04.522556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 609/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Tue Jul 5 15:54:04 2022] POST /identity/v3/users => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:54:06.558525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d724a-5080-4a7a-8afd-7e54d0ca09dc None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:06.559044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d724a-5080-4a7a-8afd-7e54d0ca09dc None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:06.559044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d724a-5080-4a7a-8afd-7e54d0ca09dc None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:06.560960 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 597/1206] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:06.565973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933d0f-2ecc-4fe9-a28f-7f8a015da8cc None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:06.566415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933d0f-2ecc-4fe9-a28f-7f8a015da8cc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:06.566518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933d0f-2ecc-4fe9-a28f-7f8a015da8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:06.582975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-88933d0f-2ecc-4fe9-a28f-7f8a015da8cc None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:06.584139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-88933d0f-2ecc-4fe9-a28f-7f8a015da8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:06.654484 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 610/1207] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:06.677767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d724a-5080-4a7a-8afd-7e54d0ca09dc None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:06.779527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d724a-5080-4a7a-8afd-7e54d0ca09dc None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:06.812791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d5a4f-cb0f-40eb-94d8-8c2d99f7957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:06.813558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d5a4f-cb0f-40eb-94d8-8c2d99f7957c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:06.813838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d5a4f-cb0f-40eb-94d8-8c2d99f7957c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:06.814108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d5a4f-cb0f-40eb-94d8-8c2d99f7957c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:06.824133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d5a4f-cb0f-40eb-94d8-8c2d99f7957c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-286d5a4f-cb0f-40eb-94d8-8c2d99f7957c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:54:06.831903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:06.833062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:06.834096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 598/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:54:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:06.839223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88933d0f-2ecc-4fe9-a28f-7f8a015da8cc None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:06.880895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:06.938223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88933d0f-2ecc-4fe9-a28f-7f8a015da8cc None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:06.971217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a848d64-85e5-4ced-a5bc-00662ad48df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:06.973933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848d64-85e5-4ced-a5bc-00662ad48df4 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:06.973933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848d64-85e5-4ced-a5bc-00662ad48df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:06.973933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848d64-85e5-4ced-a5bc-00662ad48df4 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:06.973933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a848d64-85e5-4ced-a5bc-00662ad48df4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:06.973933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a848d64-85e5-4ced-a5bc-00662ad48df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:06.974838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a848d64-85e5-4ced-a5bc-00662ad48df4 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:06.983302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 611/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:54:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:06.989055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d5a4f-cb0f-40eb-94d8-8c2d99f7957c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:07.037542 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:07.085286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d5a4f-cb0f-40eb-94d8-8c2d99f7957c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:07.118457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25707665-551d-4477-960d-2068c20b657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:07.119231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25707665-551d-4477-960d-2068c20b657f None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:07.119513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25707665-551d-4477-960d-2068c20b657f None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:07.119788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25707665-551d-4477-960d-2068c20b657f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:07.130654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25707665-551d-4477-960d-2068c20b657f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:07.132091 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25707665-551d-4477-960d-2068c20b657f None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-25707665-551d-4477-960d-2068c20b657f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:07.149054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:07.150146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:07.151129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:07.151129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:07.151129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:07.151129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:07.151129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 599/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:07.156651 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a848d64-85e5-4ced-a5bc-00662ad48df4 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:07.264152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:07.266974 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a848d64-85e5-4ced-a5bc-00662ad48df4 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:07.301874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc8885-6fc6-444a-b789-675bcd53f959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:07.302530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8885-6fc6-444a-b789-675bcd53f959 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:07.302810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8885-6fc6-444a-b789-675bcd53f959 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:07.303086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8885-6fc6-444a-b789-675bcd53f959 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:07.303793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc8885-6fc6-444a-b789-675bcd53f959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:07.304095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc8885-6fc6-444a-b789-675bcd53f959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:07.305482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc8885-6fc6-444a-b789-675bcd53f959 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:07.316669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 612/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:07.322313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25707665-551d-4477-960d-2068c20b657f None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:07.420712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25707665-551d-4477-960d-2068c20b657f None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:07.422222 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:07.454604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4343229-34bd-438c-a668-dbc87d920545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:07.455388 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343229-34bd-438c-a668-dbc87d920545 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:07.455743 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343229-34bd-438c-a668-dbc87d920545 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:07.455938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343229-34bd-438c-a668-dbc87d920545 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:07.462697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4343229-34bd-438c-a668-dbc87d920545 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:07.464104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4343229-34bd-438c-a668-dbc87d920545 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-e4343229-34bd-438c-a668-dbc87d920545 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:54:07.479310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:07.480490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:07.482275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 600/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:07.486335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc8885-6fc6-444a-b789-675bcd53f959 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:07.602288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc8885-6fc6-444a-b789-675bcd53f959 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:07.635981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd888753-aa64-4837-b882-d04274b77bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:07.636647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd888753-aa64-4837-b882-d04274b77bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:07.636969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd888753-aa64-4837-b882-d04274b77bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:07.637246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd888753-aa64-4837-b882-d04274b77bb1 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:07.638085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd888753-aa64-4837-b882-d04274b77bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:07.639001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd888753-aa64-4837-b882-d04274b77bb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:07.640536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd888753-aa64-4837-b882-d04274b77bb1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:07.657641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 613/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Tue Jul 5 15:54:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:07.663436 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4343229-34bd-438c-a668-dbc87d920545 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:07.763855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4343229-34bd-438c-a668-dbc87d920545 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:07.798665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34beca88-eaf6-442a-81cf-47ce2ddf6943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:07.799416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34beca88-eaf6-442a-81cf-47ce2ddf6943 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:07.799701 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34beca88-eaf6-442a-81cf-47ce2ddf6943 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:07.799978 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34beca88-eaf6-442a-81cf-47ce2ddf6943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:07.829095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:07.877591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34beca88-eaf6-442a-81cf-47ce2ddf6943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:07.879153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34beca88-eaf6-442a-81cf-47ce2ddf6943 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:07.981143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 601/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:54:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:54:07.987222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd888753-aa64-4837-b882-d04274b77bb1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:08.005999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:08.085342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd888753-aa64-4837-b882-d04274b77bb1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:08.118874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8196f-56ff-4cce-9948-fc15ec94b279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:08.119651 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8196f-56ff-4cce-9948-fc15ec94b279 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:08.119935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8196f-56ff-4cce-9948-fc15ec94b279 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:08.120218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8196f-56ff-4cce-9948-fc15ec94b279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:08.130114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8196f-56ff-4cce-9948-fc15ec94b279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:08.131550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8196f-56ff-4cce-9948-fc15ec94b279 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4af8196f-56ff-4cce-9948-fc15ec94b279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:08.149053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:08.150413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:08.151526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:08.151526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:08.151526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:08.151526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:08.151526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 614/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:08.156779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34beca88-eaf6-442a-81cf-47ce2ddf6943 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:08.221581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:08.258513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34beca88-eaf6-442a-81cf-47ce2ddf6943 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:08.291732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6b04b-4960-4daa-9fde-044b64226a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:08.292385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6b04b-4960-4daa-9fde-044b64226a3d None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:08.292666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6b04b-4960-4daa-9fde-044b64226a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:08.292963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6b04b-4960-4daa-9fde-044b64226a3d None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:08.293676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6b04b-4960-4daa-9fde-044b64226a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:08.293975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6b04b-4960-4daa-9fde-044b64226a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:08.295314 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6b04b-4960-4daa-9fde-044b64226a3d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:08.308508 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 602/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:08.314616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8196f-56ff-4cce-9948-fc15ec94b279 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:08.382968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:08.414246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8196f-56ff-4cce-9948-fc15ec94b279 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:08.447753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99781e-0858-4593-af57-15633a1adace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:08.448592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99781e-0858-4593-af57-15633a1adace None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:08.448993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99781e-0858-4593-af57-15633a1adace None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:08.449275 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99781e-0858-4593-af57-15633a1adace None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:08.456633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99781e-0858-4593-af57-15633a1adace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:08.458139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99781e-0858-4593-af57-15633a1adace None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ac99781e-0858-4593-af57-15633a1adace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:54:08.472006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:08.473475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:08.474860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:54:08.474860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:54:08.474860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:54:08.474860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:54:08.474860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:54:08.474860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:54:08.474860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:08.474860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:08.474860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 615/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:54:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:08.479841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6b04b-4960-4daa-9fde-044b64226a3d None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:08.580754 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6b04b-4960-4daa-9fde-044b64226a3d None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:08.602188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:08.613903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a759ae-9e91-4d6b-ba2e-3086119a624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:08.614551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a759ae-9e91-4d6b-ba2e-3086119a624b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:08.614882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a759ae-9e91-4d6b-ba2e-3086119a624b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:08.615163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a759ae-9e91-4d6b-ba2e-3086119a624b None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:08.615945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a759ae-9e91-4d6b-ba2e-3086119a624b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:08.616256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a759ae-9e91-4d6b-ba2e-3086119a624b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:08.617708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a759ae-9e91-4d6b-ba2e-3086119a624b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:08.630155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 603/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:08.636198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99781e-0858-4593-af57-15633a1adace None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:08.733154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99781e-0858-4593-af57-15633a1adace None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:08.761137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:08.766953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a00738-f6e8-4e90-b233-31687d3da3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:08.767596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a00738-f6e8-4e90-b233-31687d3da3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:08.767879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a00738-f6e8-4e90-b233-31687d3da3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:08.768154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a00738-f6e8-4e90-b233-31687d3da3a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:08.769005 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a00738-f6e8-4e90-b233-31687d3da3a3 None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=75b7f541993b4a7c915c9e6d440e2674) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:08.769317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a00738-f6e8-4e90-b233-31687d3da3a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:08.770812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a00738-f6e8-4e90-b233-31687d3da3a3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:08.776086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 616/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:54:08 2022] GET /identity/v3/role_assignments?user.id=75b7f541993b4a7c915c9e6d440e2674&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:10.757410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a63e7-9a18-46cc-bd75-e67c84089afd None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:10.757993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a63e7-9a18-46cc-bd75-e67c84089afd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:10.757993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a63e7-9a18-46cc-bd75-e67c84089afd None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:10.759291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 604/1220] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:10.764344 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb3c39-68a4-4181-8427-ed788e77e6ae None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:10.764803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb3c39-68a4-4181-8427-ed788e77e6ae None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:10.764933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb3c39-68a4-4181-8427-ed788e77e6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:10.782031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b2fb3c39-68a4-4181-8427-ed788e77e6ae None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:10.783213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b2fb3c39-68a4-4181-8427-ed788e77e6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:10.849278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 617/1221] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:10.871797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a63e7-9a18-46cc-bd75-e67c84089afd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:10.970838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a63e7-9a18-46cc-bd75-e67c84089afd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:11.003081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067265ea-f973-48fd-b0bd-5fff6c8672e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:11.003852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067265ea-f973-48fd-b0bd-5fff6c8672e8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:11.004128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067265ea-f973-48fd-b0bd-5fff6c8672e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:11.004395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067265ea-f973-48fd-b0bd-5fff6c8672e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:11.014676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067265ea-f973-48fd-b0bd-5fff6c8672e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-067265ea-f973-48fd-b0bd-5fff6c8672e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:54:11.022177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:11.029190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:11.030735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 605/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:54:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:11.031523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb3c39-68a4-4181-8427-ed788e77e6ae None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:11.125392 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb3c39-68a4-4181-8427-ed788e77e6ae None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:11.148114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:11.158974 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66339d-6197-4d3f-a416-ff0d83cff5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:11.159855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66339d-6197-4d3f-a416-ff0d83cff5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:11.159982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66339d-6197-4d3f-a416-ff0d83cff5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:11.160159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66339d-6197-4d3f-a416-ff0d83cff5a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:11.160938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66339d-6197-4d3f-a416-ff0d83cff5a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:11.161243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66339d-6197-4d3f-a416-ff0d83cff5a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:11.162606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66339d-6197-4d3f-a416-ff0d83cff5a1 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:11.170374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 618/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:54:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:11.175950 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067265ea-f973-48fd-b0bd-5fff6c8672e8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:11.219305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:11.274817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067265ea-f973-48fd-b0bd-5fff6c8672e8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:11.308010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27889c3-4945-4175-a9c7-a2f67c095bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:11.308816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27889c3-4945-4175-a9c7-a2f67c095bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:11.309098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27889c3-4945-4175-a9c7-a2f67c095bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:11.309373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27889c3-4945-4175-a9c7-a2f67c095bb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:11.320170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27889c3-4945-4175-a9c7-a2f67c095bb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:11.321640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27889c3-4945-4175-a9c7-a2f67c095bb7 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-f27889c3-4945-4175-a9c7-a2f67c095bb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:11.338635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:11.339706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:11.340642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:11.340642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:11.340642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:11.340642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:11.340642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 606/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:11.345681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66339d-6197-4d3f-a416-ff0d83cff5a1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:11.442062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66339d-6197-4d3f-a416-ff0d83cff5a1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:11.477452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7348c-2753-48e7-a885-5c7dc4967146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:11.478099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7348c-2753-48e7-a885-5c7dc4967146 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:11.478379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7348c-2753-48e7-a885-5c7dc4967146 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:11.478653 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7348c-2753-48e7-a885-5c7dc4967146 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:11.479362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7348c-2753-48e7-a885-5c7dc4967146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:11.479663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7348c-2753-48e7-a885-5c7dc4967146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:11.481047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7348c-2753-48e7-a885-5c7dc4967146 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:11.492594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 619/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:11.498831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27889c3-4945-4175-a9c7-a2f67c095bb7 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:11.591171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:11.597752 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27889c3-4945-4175-a9c7-a2f67c095bb7 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:11.633377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30366600-e148-41f9-8983-17991be31330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:11.634168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30366600-e148-41f9-8983-17991be31330 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:11.634447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30366600-e148-41f9-8983-17991be31330 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:11.634717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30366600-e148-41f9-8983-17991be31330 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:11.641541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30366600-e148-41f9-8983-17991be31330 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:11.643014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30366600-e148-41f9-8983-17991be31330 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-30366600-e148-41f9-8983-17991be31330 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:54:11.657927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:11.659154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:11.660575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 607/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:11.665191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7348c-2753-48e7-a885-5c7dc4967146 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:11.712124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:11.765687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7348c-2753-48e7-a885-5c7dc4967146 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:11.798214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82248ae6-bd37-4d0e-9f81-fe7d2a1b2aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:11.798983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82248ae6-bd37-4d0e-9f81-fe7d2a1b2aae None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:11.799267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82248ae6-bd37-4d0e-9f81-fe7d2a1b2aae None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:11.799545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82248ae6-bd37-4d0e-9f81-fe7d2a1b2aae None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:11.800375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82248ae6-bd37-4d0e-9f81-fe7d2a1b2aae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:11.800685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82248ae6-bd37-4d0e-9f81-fe7d2a1b2aae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:11.802134 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82248ae6-bd37-4d0e-9f81-fe7d2a1b2aae None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:11.819978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 620/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Tue Jul 5 15:54:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:11.825656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30366600-e148-41f9-8983-17991be31330 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:11.923664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30366600-e148-41f9-8983-17991be31330 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:11.957154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff495a46-a2d1-4c95-b631-f86c7eb3dac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:11.957958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff495a46-a2d1-4c95-b631-f86c7eb3dac8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:11.958239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff495a46-a2d1-4c95-b631-f86c7eb3dac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:11.958513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff495a46-a2d1-4c95-b631-f86c7eb3dac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:11.968799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff495a46-a2d1-4c95-b631-f86c7eb3dac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:11.970350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff495a46-a2d1-4c95-b631-f86c7eb3dac8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-ff495a46-a2d1-4c95-b631-f86c7eb3dac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:11.988138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:11.989245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:11.990366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:11.990366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:11.990366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:11.990366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:11.990366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 608/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:11.995009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82248ae6-bd37-4d0e-9f81-fe7d2a1b2aae None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:12.091180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82248ae6-bd37-4d0e-9f81-fe7d2a1b2aae None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:12.094061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:12.125077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a3b65-920a-4143-8ac4-0afdf8cb49cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:12.125735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a3b65-920a-4143-8ac4-0afdf8cb49cf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:12.126021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a3b65-920a-4143-8ac4-0afdf8cb49cf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:12.126299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a3b65-920a-4143-8ac4-0afdf8cb49cf None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:12.127021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a3b65-920a-4143-8ac4-0afdf8cb49cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:12.127327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a3b65-920a-4143-8ac4-0afdf8cb49cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:12.128697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a3b65-920a-4143-8ac4-0afdf8cb49cf None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:12.139664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 621/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:12.145590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff495a46-a2d1-4c95-b631-f86c7eb3dac8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:12.154875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:12.243512 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff495a46-a2d1-4c95-b631-f86c7eb3dac8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:12.276485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07465d78-1e6c-444e-91fe-42f0ccd4f508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:12.277307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07465d78-1e6c-444e-91fe-42f0ccd4f508 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:12.277585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07465d78-1e6c-444e-91fe-42f0ccd4f508 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:12.282474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07465d78-1e6c-444e-91fe-42f0ccd4f508 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:12.284821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07465d78-1e6c-444e-91fe-42f0ccd4f508 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:12.286233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07465d78-1e6c-444e-91fe-42f0ccd4f508 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-07465d78-1e6c-444e-91fe-42f0ccd4f508 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:54:12.300735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:12.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:12.302942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:54:12.302942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:54:12.302942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:54:12.302942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:54:12.302942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:54:12.302942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:54:12.302942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:12.302942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:12.302942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 609/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:54:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:12.307695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a3b65-920a-4143-8ac4-0afdf8cb49cf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:12.406856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a3b65-920a-4143-8ac4-0afdf8cb49cf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:12.438940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e902483-93c0-4aa1-8619-345e8930e61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:12.439589 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e902483-93c0-4aa1-8619-345e8930e61b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:12.439874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e902483-93c0-4aa1-8619-345e8930e61b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:12.440150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e902483-93c0-4aa1-8619-345e8930e61b None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:12.440986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e902483-93c0-4aa1-8619-345e8930e61b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:12.441326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e902483-93c0-4aa1-8619-345e8930e61b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:12.442751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e902483-93c0-4aa1-8619-345e8930e61b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:12.454956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 622/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:12.461078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07465d78-1e6c-444e-91fe-42f0ccd4f508 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:12.524661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:12.559534 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07465d78-1e6c-444e-91fe-42f0ccd4f508 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:12.593291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d6d39-54e4-4767-8da7-2fce7851289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:12.594155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d6d39-54e4-4767-8da7-2fce7851289d None admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:12.594435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d6d39-54e4-4767-8da7-2fce7851289d None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:12.594708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d6d39-54e4-4767-8da7-2fce7851289d None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/75b7f541993b4a7c915c9e6d440e2674/roles/ac08a857de8b484fb5dfc65cd4cdc044` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:12.609226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d6d39-54e4-4767-8da7-2fce7851289d None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=75b7f541993b4a7c915c9e6d440e2674, role_id=ac08a857de8b484fb5dfc65cd4cdc044)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:12.610704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d6d39-54e4-4767-8da7-2fce7851289d None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:12.707147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 610/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:54:12 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/75b7f541993b4a7c915c9e6d440e2674/roles/ac08a857de8b484fb5dfc65cd4cdc044 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:54:14.712265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3af7-4a19-4171-bb80-6ca2e54566de None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:14.712805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3af7-4a19-4171-bb80-6ca2e54566de None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:14.712914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3af7-4a19-4171-bb80-6ca2e54566de None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:14.714255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 623/1233] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:14.720031 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ad3e1-5871-4c9f-97be-6f18cd7b8c8d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:14.720428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ad3e1-5871-4c9f-97be-6f18cd7b8c8d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:14.720579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ad3e1-5871-4c9f-97be-6f18cd7b8c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:14.737343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-cc7ad3e1-5871-4c9f-97be-6f18cd7b8c8d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:14.738548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-cc7ad3e1-5871-4c9f-97be-6f18cd7b8c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:14.803339 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 611/1234] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:14.827097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3af7-4a19-4171-bb80-6ca2e54566de None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:14.853122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:14.927003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3af7-4a19-4171-bb80-6ca2e54566de None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:14.959486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee180e81-b434-4e17-a084-94c50cf5bf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:14.960293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee180e81-b434-4e17-a084-94c50cf5bf8c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:14.960576 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee180e81-b434-4e17-a084-94c50cf5bf8c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:14.960879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee180e81-b434-4e17-a084-94c50cf5bf8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:14.973531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee180e81-b434-4e17-a084-94c50cf5bf8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ee180e81-b434-4e17-a084-94c50cf5bf8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:54:14.979725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:14.980890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:14.982072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 624/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Tue Jul 5 15:54:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:14.986947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ad3e1-5871-4c9f-97be-6f18cd7b8c8d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:15.036852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:15.087718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ad3e1-5871-4c9f-97be-6f18cd7b8c8d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:15.120226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19b98e-9961-4622-b62d-7e203117c395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:15.120910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19b98e-9961-4622-b62d-7e203117c395 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:15.121196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19b98e-9961-4622-b62d-7e203117c395 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:15.121477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19b98e-9961-4622-b62d-7e203117c395 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:15.122218 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19b98e-9961-4622-b62d-7e203117c395 None admin] RBAC: Adding query filter params (name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:15.122536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19b98e-9961-4622-b62d-7e203117c395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:15.124410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19b98e-9961-4622-b62d-7e203117c395 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:15.132016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 612/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Tue Jul 5 15:54:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:15.137686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee180e81-b434-4e17-a084-94c50cf5bf8c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:15.223868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:15.240423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee180e81-b434-4e17-a084-94c50cf5bf8c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:15.273740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0c25e-d011-4393-a7d2-3aa40bd1d07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:15.274529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0c25e-d011-4393-a7d2-3aa40bd1d07c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:15.274816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0c25e-d011-4393-a7d2-3aa40bd1d07c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:15.275092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0c25e-d011-4393-a7d2-3aa40bd1d07c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:15.285554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0c25e-d011-4393-a7d2-3aa40bd1d07c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:15.287032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0c25e-d011-4393-a7d2-3aa40bd1d07c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-4da0c25e-d011-4393-a7d2-3aa40bd1d07c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:15.305670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:15.306714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:15.307727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:15.307727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:15.307727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:15.307727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:15.307727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 625/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:15.312601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19b98e-9961-4622-b62d-7e203117c395 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:15.412018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19b98e-9961-4622-b62d-7e203117c395 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:15.420679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:15.444799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62afde0-220f-4730-8f41-99c4304d942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:15.445445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62afde0-220f-4730-8f41-99c4304d942e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:15.445729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62afde0-220f-4730-8f41-99c4304d942e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:15.446005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62afde0-220f-4730-8f41-99c4304d942e None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:15.446723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62afde0-220f-4730-8f41-99c4304d942e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:15.447033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62afde0-220f-4730-8f41-99c4304d942e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:15.448394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62afde0-220f-4730-8f41-99c4304d942e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:15.464805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 613/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:15.465991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0c25e-d011-4393-a7d2-3aa40bd1d07c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:15.564240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0c25e-d011-4393-a7d2-3aa40bd1d07c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:15.598731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790482e7-1fa7-430f-8528-e26a740972bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:15.599518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790482e7-1fa7-430f-8528-e26a740972bc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:15.599800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790482e7-1fa7-430f-8528-e26a740972bc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:15.600075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790482e7-1fa7-430f-8528-e26a740972bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:15.603239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:15.607734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790482e7-1fa7-430f-8528-e26a740972bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:15.609421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790482e7-1fa7-430f-8528-e26a740972bc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-790482e7-1fa7-430f-8528-e26a740972bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:54:15.629280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:15.630435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:15.631621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 626/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:15.636567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62afde0-220f-4730-8f41-99c4304d942e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:15.735826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62afde0-220f-4730-8f41-99c4304d942e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:15.770006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d33b3-6558-403e-8b72-3dd23fb4c498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:15.770668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d33b3-6558-403e-8b72-3dd23fb4c498 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:15.770954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d33b3-6558-403e-8b72-3dd23fb4c498 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:15.771233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d33b3-6558-403e-8b72-3dd23fb4c498 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:15.772077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d33b3-6558-403e-8b72-3dd23fb4c498 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:15.772392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d33b3-6558-403e-8b72-3dd23fb4c498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:15.773846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d33b3-6558-403e-8b72-3dd23fb4c498 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:15.790524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 614/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Tue Jul 5 15:54:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:15.796184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790482e7-1fa7-430f-8528-e26a740972bc None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:15.894018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790482e7-1fa7-430f-8528-e26a740972bc None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:15.926417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89262e0-da81-4c84-b66a-df316e58f4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:15.927049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89262e0-da81-4c84-b66a-df316e58f4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:15.927330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89262e0-da81-4c84-b66a-df316e58f4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:15.927604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89262e0-da81-4c84-b66a-df316e58f4f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:15.987831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:15.994542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89262e0-da81-4c84-b66a-df316e58f4f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:15.995997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89262e0-da81-4c84-b66a-df316e58f4f6 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:16.092727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 627/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:54:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:54:16.098485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d33b3-6558-403e-8b72-3dd23fb4c498 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:16.126067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:16.194268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d33b3-6558-403e-8b72-3dd23fb4c498 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:16.226361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a915a-4683-49cc-a962-d7918a42e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:16.227414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a915a-4683-49cc-a962-d7918a42e025 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:16.227702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a915a-4683-49cc-a962-d7918a42e025 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:16.227983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a915a-4683-49cc-a962-d7918a42e025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:16.239099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a915a-4683-49cc-a962-d7918a42e025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:16.241488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a915a-4683-49cc-a962-d7918a42e025 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-7c3a915a-4683-49cc-a962-d7918a42e025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:16.258176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:16.259092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:16.259991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:16.259991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:16.259991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:16.259991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:16.259991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 615/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:16.265120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89262e0-da81-4c84-b66a-df316e58f4f6 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:16.358062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:16.363613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89262e0-da81-4c84-b66a-df316e58f4f6 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:16.396231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e6410-32cf-4760-ad86-51f1dec51331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:16.396911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e6410-32cf-4760-ad86-51f1dec51331 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:16.397192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e6410-32cf-4760-ad86-51f1dec51331 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:16.397466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e6410-32cf-4760-ad86-51f1dec51331 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:16.398181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e6410-32cf-4760-ad86-51f1dec51331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:16.398484 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e6410-32cf-4760-ad86-51f1dec51331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:16.399841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e6410-32cf-4760-ad86-51f1dec51331 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:16.411053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 628/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:16.416579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a915a-4683-49cc-a962-d7918a42e025 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:16.493500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:16.513989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a915a-4683-49cc-a962-d7918a42e025 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:16.547330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3dcb7-0068-4e5b-b8bf-25db833708d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:16.548104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3dcb7-0068-4e5b-b8bf-25db833708d9 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:16.548385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3dcb7-0068-4e5b-b8bf-25db833708d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:16.548660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3dcb7-0068-4e5b-b8bf-25db833708d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:16.555007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3dcb7-0068-4e5b-b8bf-25db833708d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:16.556416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3dcb7-0068-4e5b-b8bf-25db833708d9 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-1ac3dcb7-0068-4e5b-b8bf-25db833708d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:54:16.569964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:16.570912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:16.581005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:54:16.581005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:54:16.581005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:54:16.581005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:54:16.581005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:54:16.581005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:54:16.581005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:16.581005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:16.581005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 616/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:16.581722 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e6410-32cf-4760-ad86-51f1dec51331 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:16.675976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e6410-32cf-4760-ad86-51f1dec51331 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:16.708577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ecbf0-f158-4821-972d-26a53183c364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:16.709253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ecbf0-f158-4821-972d-26a53183c364 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:16.709534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ecbf0-f158-4821-972d-26a53183c364 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:16.709805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ecbf0-f158-4821-972d-26a53183c364 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:16.710638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ecbf0-f158-4821-972d-26a53183c364 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:16.710914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ecbf0-f158-4821-972d-26a53183c364 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:16.712338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ecbf0-f158-4821-972d-26a53183c364 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:16.723953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 629/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:16.730188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3dcb7-0068-4e5b-b8bf-25db833708d9 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:16.827204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3dcb7-0068-4e5b-b8bf-25db833708d9 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:16.859329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd5a77-2a22-4715-ac58-41ffd067fa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:16.859961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd5a77-2a22-4715-ac58-41ffd067fa47 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:16.860246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd5a77-2a22-4715-ac58-41ffd067fa47 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:16.860517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd5a77-2a22-4715-ac58-41ffd067fa47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:16.861365 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd5a77-2a22-4715-ac58-41ffd067fa47 None admin] RBAC: Adding query filter params (role.id=ac08a857de8b484fb5dfc65cd4cdc044, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=75b7f541993b4a7c915c9e6d440e2674) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:16.861676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd5a77-2a22-4715-ac58-41ffd067fa47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:16.863078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd5a77-2a22-4715-ac58-41ffd067fa47 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:16.868833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 617/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:54:16 2022] GET /identity/v3/role_assignments?user.id=75b7f541993b4a7c915c9e6d440e2674&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=ac08a857de8b484fb5dfc65cd4cdc044 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:18.965616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926852d-ed35-4d6d-a656-ea7fcae126a2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:18.966155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926852d-ed35-4d6d-a656-ea7fcae126a2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:18.966155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926852d-ed35-4d6d-a656-ea7fcae126a2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:18.967566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 630/1247] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:18.972575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f38fb-a962-402e-9222-ed10e667e4cb None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:18.973022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f38fb-a962-402e-9222-ed10e667e4cb None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:18.973173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f38fb-a962-402e-9222-ed10e667e4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:18.978398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:18.990358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e97f38fb-a962-402e-9222-ed10e667e4cb None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:18.992529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e97f38fb-a962-402e-9222-ed10e667e4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:19.057023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 618/1248] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:19.081641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926852d-ed35-4d6d-a656-ea7fcae126a2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:19.099761 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:19.182122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926852d-ed35-4d6d-a656-ea7fcae126a2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:19.215577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce132ad-2468-41c8-9641-08f6a37f08bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:19.216494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce132ad-2468-41c8-9641-08f6a37f08bf None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:19.216629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce132ad-2468-41c8-9641-08f6a37f08bf None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:19.216926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce132ad-2468-41c8-9641-08f6a37f08bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:19.231943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce132ad-2468-41c8-9641-08f6a37f08bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0ce132ad-2468-41c8-9641-08f6a37f08bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:54:19.242129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:19.243303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:19.244535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 631/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:54:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:19.250087 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f38fb-a962-402e-9222-ed10e667e4cb None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:19.347864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f38fb-a962-402e-9222-ed10e667e4cb None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:19.377367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:19.382203 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab53779-4b21-4c95-9aea-9ec1d818bf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:19.382863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab53779-4b21-4c95-9aea-9ec1d818bf7e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:19.383213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab53779-4b21-4c95-9aea-9ec1d818bf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:19.383485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab53779-4b21-4c95-9aea-9ec1d818bf7e None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:19.384227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab53779-4b21-4c95-9aea-9ec1d818bf7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:19.384534 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab53779-4b21-4c95-9aea-9ec1d818bf7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:19.385925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab53779-4b21-4c95-9aea-9ec1d818bf7e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:19.393969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 619/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:54:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:19.399830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce132ad-2468-41c8-9641-08f6a37f08bf None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:19.478258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:19.500858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce132ad-2468-41c8-9641-08f6a37f08bf None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:19.533324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbda77-750b-4f45-af87-d265ea06d41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:19.534105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbda77-750b-4f45-af87-d265ea06d41b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:19.534416 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbda77-750b-4f45-af87-d265ea06d41b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:19.534687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbda77-750b-4f45-af87-d265ea06d41b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:19.545404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbda77-750b-4f45-af87-d265ea06d41b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:19.546890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbda77-750b-4f45-af87-d265ea06d41b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-c1fbda77-750b-4f45-af87-d265ea06d41b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:19.563814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:19.564717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:19.565799 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:19.565799 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:19.565799 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:19.565799 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:19.565799 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 632/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:54:19.570986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab53779-4b21-4c95-9aea-9ec1d818bf7e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:19.674406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab53779-4b21-4c95-9aea-9ec1d818bf7e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:19.706201 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07757939-ec6b-4c7f-936e-2e6a32a49b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:19.706848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07757939-ec6b-4c7f-936e-2e6a32a49b8b None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:19.707130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07757939-ec6b-4c7f-936e-2e6a32a49b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:19.707403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07757939-ec6b-4c7f-936e-2e6a32a49b8b None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:19.708115 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07757939-ec6b-4c7f-936e-2e6a32a49b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:19.708417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07757939-ec6b-4c7f-936e-2e6a32a49b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:19.709800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07757939-ec6b-4c7f-936e-2e6a32a49b8b None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:19.721599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 620/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:19.727934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbda77-750b-4f45-af87-d265ea06d41b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:19.824839 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbda77-750b-4f45-af87-d265ea06d41b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:19.850091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:19.858448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8877ca-4c15-4ccd-8602-bba69604499a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:19.859227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8877ca-4c15-4ccd-8602-bba69604499a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:19.859510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8877ca-4c15-4ccd-8602-bba69604499a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:19.859785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8877ca-4c15-4ccd-8602-bba69604499a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:19.866978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8877ca-4c15-4ccd-8602-bba69604499a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:19.868409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8877ca-4c15-4ccd-8602-bba69604499a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-0d8877ca-4c15-4ccd-8602-bba69604499a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:54:19.882229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:19.883310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:19.884477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 633/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:54:19.891782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07757939-ec6b-4c7f-936e-2e6a32a49b8b None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:19.945566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:19.988934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07757939-ec6b-4c7f-936e-2e6a32a49b8b None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:20.021022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db0593-3b17-40cd-ac3b-e313ee39dff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:20.021680 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db0593-3b17-40cd-ac3b-e313ee39dff3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:20.021983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db0593-3b17-40cd-ac3b-e313ee39dff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:20.022254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db0593-3b17-40cd-ac3b-e313ee39dff3 None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:20.023097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db0593-3b17-40cd-ac3b-e313ee39dff3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:20.023405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db0593-3b17-40cd-ac3b-e313ee39dff3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:20.024907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db0593-3b17-40cd-ac3b-e313ee39dff3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:20.042518 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 621/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Tue Jul 5 15:54:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:20.048439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8877ca-4c15-4ccd-8602-bba69604499a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:20.147340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8877ca-4c15-4ccd-8602-bba69604499a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:20.180089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9094e-8f7b-4b7a-be66-ff19920a0be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:20.180732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9094e-8f7b-4b7a-be66-ff19920a0be0 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:20.181049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9094e-8f7b-4b7a-be66-ff19920a0be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:20.181328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9094e-8f7b-4b7a-be66-ff19920a0be0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:20.236071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:20.248393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9094e-8f7b-4b7a-be66-ff19920a0be0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:20.249874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9094e-8f7b-4b7a-be66-ff19920a0be0 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:20.345260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 634/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:54:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:54:20.351172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db0593-3b17-40cd-ac3b-e313ee39dff3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:20.448679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db0593-3b17-40cd-ac3b-e313ee39dff3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:20.461472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:20.482865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7c985-f710-4d8a-ac6a-4595c53840a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:20.483641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7c985-f710-4d8a-ac6a-4595c53840a3 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:20.483922 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7c985-f710-4d8a-ac6a-4595c53840a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:20.484192 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7c985-f710-4d8a-ac6a-4595c53840a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:20.493823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7c985-f710-4d8a-ac6a-4595c53840a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:20.497088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7c985-f710-4d8a-ac6a-4595c53840a3 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-42f7c985-f710-4d8a-ac6a-4595c53840a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:20.512106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:20.513216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:20.514179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:20.514179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:20.514179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:20.514179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:20.514179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 622/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:20.519010 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9094e-8f7b-4b7a-be66-ff19920a0be0 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:20.626080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:20.637917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9094e-8f7b-4b7a-be66-ff19920a0be0 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:20.671209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618fe84-94da-40ba-b93d-91390bd79f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:20.671857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618fe84-94da-40ba-b93d-91390bd79f43 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:20.672135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618fe84-94da-40ba-b93d-91390bd79f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:20.672408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618fe84-94da-40ba-b93d-91390bd79f43 None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:20.673152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618fe84-94da-40ba-b93d-91390bd79f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:20.676203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618fe84-94da-40ba-b93d-91390bd79f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:20.676203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618fe84-94da-40ba-b93d-91390bd79f43 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:20.685685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 635/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:20.691560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7c985-f710-4d8a-ac6a-4595c53840a3 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:20.789716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7c985-f710-4d8a-ac6a-4595c53840a3 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:20.822986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb065af0-3220-4c56-927a-3917c95cd06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:20.823773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb065af0-3220-4c56-927a-3917c95cd06c None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:20.824054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb065af0-3220-4c56-927a-3917c95cd06c None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:20.824330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb065af0-3220-4c56-927a-3917c95cd06c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:20.831581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb065af0-3220-4c56-927a-3917c95cd06c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:20.833048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb065af0-3220-4c56-927a-3917c95cd06c None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-eb065af0-3220-4c56-927a-3917c95cd06c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:54:20.847107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:20.848291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:20.849454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:54:20.849454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:54:20.849454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:54:20.849454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:54:20.849454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:54:20.849454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:54:20.849454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:20.849454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:20.849454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 623/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:54:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:20.854754 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618fe84-94da-40ba-b93d-91390bd79f43 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:20.960159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618fe84-94da-40ba-b93d-91390bd79f43 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:20.994584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af707fe-d359-46ba-9182-2d54b71d7361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:20.995241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af707fe-d359-46ba-9182-2d54b71d7361 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:20.995602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af707fe-d359-46ba-9182-2d54b71d7361 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:20.995812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af707fe-d359-46ba-9182-2d54b71d7361 None admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:20.996594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af707fe-d359-46ba-9182-2d54b71d7361 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:20.996941 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af707fe-d359-46ba-9182-2d54b71d7361 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:20.998876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af707fe-d359-46ba-9182-2d54b71d7361 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:21.010822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 636/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:54:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:21.017673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb065af0-3220-4c56-927a-3917c95cd06c None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:21.043313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:21.115390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb065af0-3220-4c56-927a-3917c95cd06c None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:21.149017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d5b37-ff70-439e-b5c8-fc05c2b2ee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:21.149647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d5b37-ff70-439e-b5c8-fc05c2b2ee5e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:21.149927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d5b37-ff70-439e-b5c8-fc05c2b2ee5e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:21.150199 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d5b37-ff70-439e-b5c8-fc05c2b2ee5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:21.151019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d5b37-ff70-439e-b5c8-fc05c2b2ee5e None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=75b7f541993b4a7c915c9e6d440e2674) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:21.151371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d5b37-ff70-439e-b5c8-fc05c2b2ee5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:21.152803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d5b37-ff70-439e-b5c8-fc05c2b2ee5e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:21.158133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 624/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:54:21 2022] GET /identity/v3/role_assignments?user.id=75b7f541993b4a7c915c9e6d440e2674&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:23.152444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebc4d4-077f-4de8-9c7d-98197ed47dfb None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:23.153011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebc4d4-077f-4de8-9c7d-98197ed47dfb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:23.153123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebc4d4-077f-4de8-9c7d-98197ed47dfb None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:23.154551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 637/1261] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:23.159757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aec6ce-0060-4bd5-8655-af2326281fbe None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:23.160183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aec6ce-0060-4bd5-8655-af2326281fbe None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:23.160306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aec6ce-0060-4bd5-8655-af2326281fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:23.177285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-74aec6ce-0060-4bd5-8655-af2326281fbe None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:23.178479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-74aec6ce-0060-4bd5-8655-af2326281fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:23.245402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 625/1262] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:23.269469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebc4d4-077f-4de8-9c7d-98197ed47dfb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:23.284522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:23.376212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebc4d4-077f-4de8-9c7d-98197ed47dfb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:23.409067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49077082-22df-47c7-aec5-37df766a85f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:23.409844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49077082-22df-47c7-aec5-37df766a85f1 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:23.410125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49077082-22df-47c7-aec5-37df766a85f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:23.410397 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49077082-22df-47c7-aec5-37df766a85f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:23.420691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49077082-22df-47c7-aec5-37df766a85f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-49077082-22df-47c7-aec5-37df766a85f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:54:23.428189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:23.429206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:23.430070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 638/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:54:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:23.435487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aec6ce-0060-4bd5-8655-af2326281fbe None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:23.453256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:23.532005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aec6ce-0060-4bd5-8655-af2326281fbe None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:23.567087 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf787c2-b96c-40d8-88a8-3791f9c66528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:23.571401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf787c2-b96c-40d8-88a8-3791f9c66528 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:23.571694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf787c2-b96c-40d8-88a8-3791f9c66528 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:23.571978 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf787c2-b96c-40d8-88a8-3791f9c66528 None admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:23.572793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf787c2-b96c-40d8-88a8-3791f9c66528 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:23.573109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf787c2-b96c-40d8-88a8-3791f9c66528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:23.574504 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf787c2-b96c-40d8-88a8-3791f9c66528 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:23.583357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 626/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:54:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:23.589192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49077082-22df-47c7-aec5-37df766a85f1 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:23.660413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:23.697950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49077082-22df-47c7-aec5-37df766a85f1 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:23.732800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd572a4-3c40-414f-b572-8492b2b6fdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:23.733625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd572a4-3c40-414f-b572-8492b2b6fdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:23.733910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd572a4-3c40-414f-b572-8492b2b6fdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:23.734233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd572a4-3c40-414f-b572-8492b2b6fdd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:23.745500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd572a4-3c40-414f-b572-8492b2b6fdd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:23.746952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd572a4-3c40-414f-b572-8492b2b6fdd5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-8fd572a4-3c40-414f-b572-8492b2b6fdd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:23.764334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:23.767436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:23.768362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:23.768362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:23.768362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:23.768362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:23.768362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 639/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:23.771470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf787c2-b96c-40d8-88a8-3791f9c66528 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:23.852855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:23.871544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf787c2-b96c-40d8-88a8-3791f9c66528 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:23.911635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b83922-56f4-4c16-b309-fca0aafa82db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:23.912214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b83922-56f4-4c16-b309-fca0aafa82db None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:23.912493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b83922-56f4-4c16-b309-fca0aafa82db None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:23.912800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b83922-56f4-4c16-b309-fca0aafa82db None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:23.913534 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b83922-56f4-4c16-b309-fca0aafa82db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:23.913838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b83922-56f4-4c16-b309-fca0aafa82db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:23.915237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b83922-56f4-4c16-b309-fca0aafa82db None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:23.926369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 627/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:23.932284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd572a4-3c40-414f-b572-8492b2b6fdd5 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:24.032975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd572a4-3c40-414f-b572-8492b2b6fdd5 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:24.052471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:24.066775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d5a40-0a39-4207-b2a1-42b6a8ff9613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:24.067571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d5a40-0a39-4207-b2a1-42b6a8ff9613 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:24.067886 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d5a40-0a39-4207-b2a1-42b6a8ff9613 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:24.068166 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d5a40-0a39-4207-b2a1-42b6a8ff9613 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:24.076511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d5a40-0a39-4207-b2a1-42b6a8ff9613 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:24.078059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d5a40-0a39-4207-b2a1-42b6a8ff9613 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-078d5a40-0a39-4207-b2a1-42b6a8ff9613 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:54:24.092994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:24.094070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:24.095211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 640/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:24.100246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b83922-56f4-4c16-b309-fca0aafa82db None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:24.209651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b83922-56f4-4c16-b309-fca0aafa82db None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:24.246964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d85b9b-800b-455e-acc5-b77d2b2d398e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:24.247656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d85b9b-800b-455e-acc5-b77d2b2d398e None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:24.247938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d85b9b-800b-455e-acc5-b77d2b2d398e None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:24.248210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d85b9b-800b-455e-acc5-b77d2b2d398e None admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:24.249072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d85b9b-800b-455e-acc5-b77d2b2d398e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:24.249377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d85b9b-800b-455e-acc5-b77d2b2d398e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:24.250803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d85b9b-800b-455e-acc5-b77d2b2d398e None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:24.259813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:24.268029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 628/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Tue Jul 5 15:54:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:24.273680 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d5a40-0a39-4207-b2a1-42b6a8ff9613 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:24.374319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d5a40-0a39-4207-b2a1-42b6a8ff9613 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:24.407673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311493db-6cee-41a3-a2e6-2031e758c30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:24.408452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311493db-6cee-41a3-a2e6-2031e758c30b None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:24.408740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311493db-6cee-41a3-a2e6-2031e758c30b None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:24.409044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311493db-6cee-41a3-a2e6-2031e758c30b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:24.419252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311493db-6cee-41a3-a2e6-2031e758c30b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:24.420708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311493db-6cee-41a3-a2e6-2031e758c30b None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-311493db-6cee-41a3-a2e6-2031e758c30b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:24.437199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:24.440400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:24.441587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:24.441587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:24.441587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:24.441587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:24.441587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 641/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:24.444133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d85b9b-800b-455e-acc5-b77d2b2d398e None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:24.546493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d85b9b-800b-455e-acc5-b77d2b2d398e None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:24.579702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b02777-1893-4f29-9bfd-a1b1a3df2f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:24.580354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b02777-1893-4f29-9bfd-a1b1a3df2f17 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:24.580636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b02777-1893-4f29-9bfd-a1b1a3df2f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:24.580940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b02777-1893-4f29-9bfd-a1b1a3df2f17 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:24.581671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b02777-1893-4f29-9bfd-a1b1a3df2f17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:24.586047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b02777-1893-4f29-9bfd-a1b1a3df2f17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:24.586047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b02777-1893-4f29-9bfd-a1b1a3df2f17 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:24.594327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 629/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:24.599946 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311493db-6cee-41a3-a2e6-2031e758c30b None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:24.633487 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:24.701528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311493db-6cee-41a3-a2e6-2031e758c30b None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:24.736419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a8b37-e511-484e-b768-fc65b518780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:24.737249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a8b37-e511-484e-b768-fc65b518780c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:24.737535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a8b37-e511-484e-b768-fc65b518780c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:24.737808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a8b37-e511-484e-b768-fc65b518780c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:24.744451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a8b37-e511-484e-b768-fc65b518780c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:24.745938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a8b37-e511-484e-b768-fc65b518780c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-ef0a8b37-e511-484e-b768-fc65b518780c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:54:24.759580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:24.760580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:24.761710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:54:24.761710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:54:24.761710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:54:24.761710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:54:24.761710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:54:24.761710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:54:24.761710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:24.761710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:24.761710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 642/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:54:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:24.766536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b02777-1893-4f29-9bfd-a1b1a3df2f17 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:24.823599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:24.863802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b02777-1893-4f29-9bfd-a1b1a3df2f17 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:24.895496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfede91-7bc4-4547-a841-cf911b5cb5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:24.896149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfede91-7bc4-4547-a841-cf911b5cb5ae None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:24.896428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfede91-7bc4-4547-a841-cf911b5cb5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:24.896700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfede91-7bc4-4547-a841-cf911b5cb5ae None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:24.897502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfede91-7bc4-4547-a841-cf911b5cb5ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:24.897812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfede91-7bc4-4547-a841-cf911b5cb5ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:24.899264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfede91-7bc4-4547-a841-cf911b5cb5ae None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:24.911297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 630/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:54:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:24.917189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a8b37-e511-484e-b768-fc65b518780c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:25.005286 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:25.013473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a8b37-e511-484e-b768-fc65b518780c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:25.046887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66670014-d105-4c43-9b26-f2dd586315c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:25.047756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66670014-d105-4c43-9b26-f2dd586315c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:25.048035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66670014-d105-4c43-9b26-f2dd586315c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:25.048305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66670014-d105-4c43-9b26-f2dd586315c5 None admin] PATH_INFO: `/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/75b7f541993b4a7c915c9e6d440e2674/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:25.050237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66670014-d105-4c43-9b26-f2dd586315c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user_id=75b7f541993b4a7c915c9e6d440e2674, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:25.051660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66670014-d105-4c43-9b26-f2dd586315c5 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:25.203380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 643/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Tue Jul 5 15:54:24 2022] PUT /identity/v3/projects/bc8da89a42c743bf8f71b3d7cf6ab6fb/users/75b7f541993b4a7c915c9e6d440e2674/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:54:27.203625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80eef8-b96d-48da-ac75-aeeb2a20f6a0 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:27.204242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80eef8-b96d-48da-ac75-aeeb2a20f6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:27.204242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80eef8-b96d-48da-ac75-aeeb2a20f6a0 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:27.205578 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 631/1274] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:27.210793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1c7c-9443-412f-b6ca-a2e64f57e427 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:27.211197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1c7c-9443-412f-b6ca-a2e64f57e427 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:27.213249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1c7c-9443-412f-b6ca-a2e64f57e427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:27.228288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1b5c1c7c-9443-412f-b6ca-a2e64f57e427 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:27.229510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1b5c1c7c-9443-412f-b6ca-a2e64f57e427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:27.295527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 644/1275] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:27.319997 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80eef8-b96d-48da-ac75-aeeb2a20f6a0 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:27.421174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80eef8-b96d-48da-ac75-aeeb2a20f6a0 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:27.440416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:27.456583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256b7e0-93e2-47ad-aa50-3ce8f9f23936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:27.457340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256b7e0-93e2-47ad-aa50-3ce8f9f23936 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:27.457623 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256b7e0-93e2-47ad-aa50-3ce8f9f23936 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:27.457897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256b7e0-93e2-47ad-aa50-3ce8f9f23936 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:27.467704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256b7e0-93e2-47ad-aa50-3ce8f9f23936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-b256b7e0-93e2-47ad-aa50-3ce8f9f23936 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:54:27.475268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:27.476510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:27.477818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 632/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Tue Jul 5 15:54:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:27.482643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c1c7c-9443-412f-b6ca-a2e64f57e427 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:27.562396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:27.579624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c1c7c-9443-412f-b6ca-a2e64f57e427 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:27.612464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ce71-e50b-4e2a-bba0-dfc7f1f252e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:27.613150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ce71-e50b-4e2a-bba0-dfc7f1f252e3 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:27.613430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ce71-e50b-4e2a-bba0-dfc7f1f252e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:27.613701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ce71-e50b-4e2a-bba0-dfc7f1f252e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:27.614441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6ce71-e50b-4e2a-bba0-dfc7f1f252e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:27.614744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6ce71-e50b-4e2a-bba0-dfc7f1f252e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:27.616222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6ce71-e50b-4e2a-bba0-dfc7f1f252e3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:27.625098 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 645/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Tue Jul 5 15:54:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:27.630732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256b7e0-93e2-47ad-aa50-3ce8f9f23936 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:27.734657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256b7e0-93e2-47ad-aa50-3ce8f9f23936 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:27.766585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed13440-1c8a-49e7-bcf5-407cfb1887bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:27.767368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed13440-1c8a-49e7-bcf5-407cfb1887bf None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:27.767651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed13440-1c8a-49e7-bcf5-407cfb1887bf None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:27.767925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed13440-1c8a-49e7-bcf5-407cfb1887bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:27.778284 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed13440-1c8a-49e7-bcf5-407cfb1887bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:27.779822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed13440-1c8a-49e7-bcf5-407cfb1887bf None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-fed13440-1c8a-49e7-bcf5-407cfb1887bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:27.797665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:27.798745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:27.799836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:27.799836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:27.799836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:27.799836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:27.799836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 633/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:27.804680 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ce71-e50b-4e2a-bba0-dfc7f1f252e3 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:27.902936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ce71-e50b-4e2a-bba0-dfc7f1f252e3 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:27.936404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f9b49-6e6b-4fad-b694-32f2333a786a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:27.937084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f9b49-6e6b-4fad-b694-32f2333a786a None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:27.937367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f9b49-6e6b-4fad-b694-32f2333a786a None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:27.937640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f9b49-6e6b-4fad-b694-32f2333a786a None admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:27.938356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f9b49-6e6b-4fad-b694-32f2333a786a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:27.938759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f9b49-6e6b-4fad-b694-32f2333a786a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:27.940023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f9b49-6e6b-4fad-b694-32f2333a786a None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:27.951378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 646/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:27.957060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed13440-1c8a-49e7-bcf5-407cfb1887bf None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:27.988656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:28.055247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed13440-1c8a-49e7-bcf5-407cfb1887bf None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:28.088788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f67a0-2bcd-4dee-9010-f812318f7365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:28.089571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f67a0-2bcd-4dee-9010-f812318f7365 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:28.089852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f67a0-2bcd-4dee-9010-f812318f7365 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:28.090129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f67a0-2bcd-4dee-9010-f812318f7365 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:28.097452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f67a0-2bcd-4dee-9010-f812318f7365 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:28.099342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f67a0-2bcd-4dee-9010-f812318f7365 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-288f67a0-2bcd-4dee-9010-f812318f7365 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:54:28.113631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:28.114767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:28.115893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 634/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:28.120610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f9b49-6e6b-4fad-b694-32f2333a786a None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:28.124427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:28.219109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f9b49-6e6b-4fad-b694-32f2333a786a None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:28.251151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8f3d5-c9fa-4345-b0a6-b5236a01ac77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:28.251744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8f3d5-c9fa-4345-b0a6-b5236a01ac77 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:28.252024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8f3d5-c9fa-4345-b0a6-b5236a01ac77 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:28.252298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8f3d5-c9fa-4345-b0a6-b5236a01ac77 None admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:28.253154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8f3d5-c9fa-4345-b0a6-b5236a01ac77 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:28.253460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8f3d5-c9fa-4345-b0a6-b5236a01ac77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:28.254877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8f3d5-c9fa-4345-b0a6-b5236a01ac77 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:28.272003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 647/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Tue Jul 5 15:54:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:28.277662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f67a0-2bcd-4dee-9010-f812318f7365 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:28.365242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:28.374356 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f67a0-2bcd-4dee-9010-f812318f7365 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:28.407127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50db53a-f50c-4f88-8946-a641853a81f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:28.407760 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50db53a-f50c-4f88-8946-a641853a81f5 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:28.408041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50db53a-f50c-4f88-8946-a641853a81f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:28.408316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50db53a-f50c-4f88-8946-a641853a81f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:28.474920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50db53a-f50c-4f88-8946-a641853a81f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:28.476377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50db53a-f50c-4f88-8946-a641853a81f5 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:28.563295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:28.573215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 635/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Tue Jul 5 15:54:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:54:28.579596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8f3d5-c9fa-4345-b0a6-b5236a01ac77 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:28.640662 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:28.680139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8f3d5-c9fa-4345-b0a6-b5236a01ac77 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:28.713835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768ba33-06b2-4eb6-9127-07a590f3e92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:28.714618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768ba33-06b2-4eb6-9127-07a590f3e92d None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:28.714903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768ba33-06b2-4eb6-9127-07a590f3e92d None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:28.715180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768ba33-06b2-4eb6-9127-07a590f3e92d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:28.725405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768ba33-06b2-4eb6-9127-07a590f3e92d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:28.727106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768ba33-06b2-4eb6-9127-07a590f3e92d None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-f768ba33-06b2-4eb6-9127-07a590f3e92d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:54:28.743573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:28.748311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:28.749966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:54:28.749966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:54:28.749966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:54:28.749966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:28.749966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 648/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 15:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:28.751014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50db53a-f50c-4f88-8946-a641853a81f5 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:28.851537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50db53a-f50c-4f88-8946-a641853a81f5 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:28.885254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76866ddd-84a9-4ff6-b5ee-0402bcd2a0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:28.885920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76866ddd-84a9-4ff6-b5ee-0402bcd2a0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:28.886206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76866ddd-84a9-4ff6-b5ee-0402bcd2a0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:28.886481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76866ddd-84a9-4ff6-b5ee-0402bcd2a0d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:28.887215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76866ddd-84a9-4ff6-b5ee-0402bcd2a0d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:28.887523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76866ddd-84a9-4ff6-b5ee-0402bcd2a0d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:28.888956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76866ddd-84a9-4ff6-b5ee-0402bcd2a0d5 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:28.901845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 636/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Tue Jul 5 15:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:28.907803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768ba33-06b2-4eb6-9127-07a590f3e92d None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:29.017664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768ba33-06b2-4eb6-9127-07a590f3e92d None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:29.032858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:29.053442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee7d76-6f4e-4fb9-870a-a2cf58c1e04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:29.054249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee7d76-6f4e-4fb9-870a-a2cf58c1e04c None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:29.054528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee7d76-6f4e-4fb9-870a-a2cf58c1e04c None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:29.054807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee7d76-6f4e-4fb9-870a-a2cf58c1e04c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:29.062629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee7d76-6f4e-4fb9-870a-a2cf58c1e04c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:29.064076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee7d76-6f4e-4fb9-870a-a2cf58c1e04c None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application [None req-f5ee7d76-6f4e-4fb9-870a-a2cf58c1e04c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:54:29.078923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:29.079965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:29.081528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:54:29.081528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:54:29.081528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:54:29.081528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:54:29.081528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:54:29.081528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:54:29.081528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:54:29.081528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: ERROR keystone.server.flask.application Jul 05 15:54:29.081528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 649/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Tue Jul 5 15:54:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:29.086421 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76866ddd-84a9-4ff6-b5ee-0402bcd2a0d5 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:29.140425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:29.192046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76866ddd-84a9-4ff6-b5ee-0402bcd2a0d5 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:29.228446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdbe55-e631-4f2b-add4-ce38d456e144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:29.229226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdbe55-e631-4f2b-add4-ce38d456e144 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:29.229507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdbe55-e631-4f2b-add4-ce38d456e144 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:29.229829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdbe55-e631-4f2b-add4-ce38d456e144 None admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:29.230572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdbe55-e631-4f2b-add4-ce38d456e144 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:29.230899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdbe55-e631-4f2b-add4-ce38d456e144 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:29.232309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdbe55-e631-4f2b-add4-ce38d456e144 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:29.245169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 637/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Tue Jul 5 15:54:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:29.251272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee7d76-6f4e-4fb9-870a-a2cf58c1e04c None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:29.355609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee7d76-6f4e-4fb9-870a-a2cf58c1e04c None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:29.389197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a831f0-34d9-47f1-95dd-e6dbe152db71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:29.389856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a831f0-34d9-47f1-95dd-e6dbe152db71 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:29.390132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a831f0-34d9-47f1-95dd-e6dbe152db71 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:29.390412 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a831f0-34d9-47f1-95dd-e6dbe152db71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:29.391284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a831f0-34d9-47f1-95dd-e6dbe152db71 None admin] RBAC: Adding query filter params (role.id=c124ec42c56b4f43bb474fcf5e9b89b3, scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb, user.id=75b7f541993b4a7c915c9e6d440e2674) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:29.391550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a831f0-34d9-47f1-95dd-e6dbe152db71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:29.392994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a831f0-34d9-47f1-95dd-e6dbe152db71 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:29.398712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 650/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Tue Jul 5 15:54:29 2022] GET /identity/v3/role_assignments?user.id=75b7f541993b4a7c915c9e6d440e2674&scope.project.id=bc8da89a42c743bf8f71b3d7cf6ab6fb&role.id=c124ec42c56b4f43bb474fcf5e9b89b3 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:31.421352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a9efd-0af7-491f-a6e0-ad2bcccf19c9 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:31.421865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a9efd-0af7-491f-a6e0-ad2bcccf19c9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:31.421956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a9efd-0af7-491f-a6e0-ad2bcccf19c9 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:31.423264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 638/1288] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:31.428436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd3c35-394e-4d94-9f81-586995bbb6ca None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:31.432291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd3c35-394e-4d94-9f81-586995bbb6ca None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:31.432291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd3c35-394e-4d94-9f81-586995bbb6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:31.445948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8afd3c35-394e-4d94-9f81-586995bbb6ca None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:31.447113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8afd3c35-394e-4d94-9f81-586995bbb6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:31.475728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:31.512273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 651/1289] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:31.534526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a9efd-0af7-491f-a6e0-ad2bcccf19c9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:31.634201 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a9efd-0af7-491f-a6e0-ad2bcccf19c9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:31.644596 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:31.667555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f182452-3148-419c-be21-866c505e751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:31.668339 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f182452-3148-419c-be21-866c505e751a None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:31.668636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f182452-3148-419c-be21-866c505e751a None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:31.668946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f182452-3148-419c-be21-866c505e751a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:31.669662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f182452-3148-419c-be21-866c505e751a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:31.671619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f182452-3148-419c-be21-866c505e751a None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-0f182452-3148-419c-be21-866c505e751a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:31.683397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:31.684628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:31.685735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 639/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Tue Jul 5 15:54:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:31.690619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd3c35-394e-4d94-9f81-586995bbb6ca None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:31.794339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd3c35-394e-4d94-9f81-586995bbb6ca None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:31.828484 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4da69-b983-4802-836b-641a968db48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:31.829169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4da69-b983-4802-836b-641a968db48e None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:31.829558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4da69-b983-4802-836b-641a968db48e None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:31.829843 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4da69-b983-4802-836b-641a968db48e None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:31.830547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4da69-b983-4802-836b-641a968db48e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:31.830855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4da69-b983-4802-836b-641a968db48e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:31.832218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4da69-b983-4802-836b-641a968db48e None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:31.837194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 652/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Tue Jul 5 15:54:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:31.843588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f182452-3148-419c-be21-866c505e751a None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:31.941511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f182452-3148-419c-be21-866c505e751a None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:31.975004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52e23-3de8-4c39-8832-ba3036320935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:31.975655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52e23-3de8-4c39-8832-ba3036320935 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:31.975943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52e23-3de8-4c39-8832-ba3036320935 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:31.976225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52e23-3de8-4c39-8832-ba3036320935 None admin] PATH_INFO: `/v3/services` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:31.977213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52e23-3de8-4c39-8832-ba3036320935 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:31.977568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52e23-3de8-4c39-8832-ba3036320935 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:31.978936 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52e23-3de8-4c39-8832-ba3036320935 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:31.983311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 640/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Tue Jul 5 15:54:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:33.993109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf3321-6301-4d4f-9b60-4eeac492c9b6 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:33.993644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf3321-6301-4d4f-9b60-4eeac492c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:33.993644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf3321-6301-4d4f-9b60-4eeac492c9b6 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:33.995075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 653/1293] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:34.000286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207189c-2f0a-4f2f-b685-01d2001f6be2 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:34.000688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207189c-2f0a-4f2f-b685-01d2001f6be2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:34.000878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207189c-2f0a-4f2f-b685-01d2001f6be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:34.017572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8207189c-2f0a-4f2f-b685-01d2001f6be2 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:34.018770 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8207189c-2f0a-4f2f-b685-01d2001f6be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:34.059293 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:34.097523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 641/1294] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:34.120461 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf3321-6301-4d4f-9b60-4eeac492c9b6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:34.151014 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:34.218747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf3321-6301-4d4f-9b60-4eeac492c9b6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:34.252650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38682a-7595-41c7-9c49-63ed3f9e2ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:34.253279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38682a-7595-41c7-9c49-63ed3f9e2ee3 None admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:34.253563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38682a-7595-41c7-9c49-63ed3f9e2ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:34.256757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38682a-7595-41c7-9c49-63ed3f9e2ee3 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:34.256757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38682a-7595-41c7-9c49-63ed3f9e2ee3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:34.256757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38682a-7595-41c7-9c49-63ed3f9e2ee3 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:34.358769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 654/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Tue Jul 5 15:54:34 2022] POST /identity/v3/services => generated 246 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:54:36.399269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49544b5-e0c2-4ed5-9e3b-27801ea1e91d None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:36.400002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49544b5-e0c2-4ed5-9e3b-27801ea1e91d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:36.400002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49544b5-e0c2-4ed5-9e3b-27801ea1e91d None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:36.402531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 642/1296] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:36.408303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f1471-47d5-4fe6-8c7e-fc53474210ef None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:36.408740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f1471-47d5-4fe6-8c7e-fc53474210ef None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:36.408897 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f1471-47d5-4fe6-8c7e-fc53474210ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:36.433995 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9b1f1471-47d5-4fe6-8c7e-fc53474210ef None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:36.435200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9b1f1471-47d5-4fe6-8c7e-fc53474210ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:36.508122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 655/1297] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:36.530838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49544b5-e0c2-4ed5-9e3b-27801ea1e91d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:36.630378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49544b5-e0c2-4ed5-9e3b-27801ea1e91d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:36.663450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00d51d-dd4e-4344-bb2c-e9588bc860ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:36.664235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00d51d-dd4e-4344-bb2c-e9588bc860ff None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:36.664517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00d51d-dd4e-4344-bb2c-e9588bc860ff None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:36.664813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00d51d-dd4e-4344-bb2c-e9588bc860ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:36.665458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00d51d-dd4e-4344-bb2c-e9588bc860ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:36.666827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00d51d-dd4e-4344-bb2c-e9588bc860ff None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-9d00d51d-dd4e-4344-bb2c-e9588bc860ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:36.678830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:36.680022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:36.681276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 643/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Tue Jul 5 15:54:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:36.686677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f1471-47d5-4fe6-8c7e-fc53474210ef None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:36.704512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:36.793655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f1471-47d5-4fe6-8c7e-fc53474210ef None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:36.826532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122b780-be3a-4507-8545-84a73df83a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:36.827184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122b780-be3a-4507-8545-84a73df83a63 None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:36.827471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122b780-be3a-4507-8545-84a73df83a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:36.827748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122b780-be3a-4507-8545-84a73df83a63 None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:36.828452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122b780-be3a-4507-8545-84a73df83a63 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:36.828759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122b780-be3a-4507-8545-84a73df83a63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:36.830149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122b780-be3a-4507-8545-84a73df83a63 None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:36.835264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 656/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Tue Jul 5 15:54:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:36.841301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00d51d-dd4e-4344-bb2c-e9588bc860ff None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:36.873151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:36.939469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00d51d-dd4e-4344-bb2c-e9588bc860ff None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:36.972126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee15e54-4d76-48d2-a114-80a0924e3146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:36.973876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee15e54-4d76-48d2-a114-80a0924e3146 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:36.973876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee15e54-4d76-48d2-a114-80a0924e3146 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:36.973876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee15e54-4d76-48d2-a114-80a0924e3146 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:36.974113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee15e54-4d76-48d2-a114-80a0924e3146 None admin] RBAC: Adding query filter params (interface=public, service_id=f1008e39d5e4403982ce3136cd8039e4, region_id=RegionOne) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:36.974383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee15e54-4d76-48d2-a114-80a0924e3146 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:36.975729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee15e54-4d76-48d2-a114-80a0924e3146 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:36.980984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 644/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Tue Jul 5 15:54:36 2022] GET /identity/v3/endpoints?service_id=f1008e39d5e4403982ce3136cd8039e4&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:36.987318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122b780-be3a-4507-8545-84a73df83a63 None admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:37.070669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:37.090233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122b780-be3a-4507-8545-84a73df83a63 None admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:37.124350 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec367e-66f5-4e6c-a5c6-5b2efced1dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:37.125034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec367e-66f5-4e6c-a5c6-5b2efced1dfc None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:37.125322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec367e-66f5-4e6c-a5c6-5b2efced1dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:37.125601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec367e-66f5-4e6c-a5c6-5b2efced1dfc None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:37.126284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec367e-66f5-4e6c-a5c6-5b2efced1dfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:37.127817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec367e-66f5-4e6c-a5c6-5b2efced1dfc None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:37.134560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 657/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Tue Jul 5 15:54:36 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:54:39.151818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73a50e-17cb-4ffe-8fac-15e3cdea0d66 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:39.152389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73a50e-17cb-4ffe-8fac-15e3cdea0d66 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:39.152389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73a50e-17cb-4ffe-8fac-15e3cdea0d66 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:39.153808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 645/1302] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 15:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:54:39.159277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d3fc8-4c4a-429d-8bdb-604dca5e75c7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:39.159712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d3fc8-4c4a-429d-8bdb-604dca5e75c7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:39.159849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d3fc8-4c4a-429d-8bdb-604dca5e75c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:39.176853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7d8d3fc8-4c4a-429d-8bdb-604dca5e75c7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:54:39.178038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7d8d3fc8-4c4a-429d-8bdb-604dca5e75c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:54:39.243498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 658/1303] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 15:54:39 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:54:39.266105 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73a50e-17cb-4ffe-8fac-15e3cdea0d66 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:39.361559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:39.368462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73a50e-17cb-4ffe-8fac-15e3cdea0d66 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:39.402794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed883578-5171-411a-9f28-783e5003fec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:39.403512 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed883578-5171-411a-9f28-783e5003fec8 None admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:39.403795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed883578-5171-411a-9f28-783e5003fec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:39.404106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed883578-5171-411a-9f28-783e5003fec8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:39.405333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed883578-5171-411a-9f28-783e5003fec8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:39.406793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed883578-5171-411a-9f28-783e5003fec8 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application [None req-ed883578-5171-411a-9f28-783e5003fec8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:54:39.418420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:54:39.419593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application with Lock( Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: ERROR keystone.server.flask.application Jul 05 15:54:39.420825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 646/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Tue Jul 5 15:54:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:54:39.425702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d3fc8-4c4a-429d-8bdb-604dca5e75c7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:39.501154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:39.523345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d3fc8-4c4a-429d-8bdb-604dca5e75c7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:39.556976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f5afe-1d35-415f-a44b-4e885c6bf9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:39.557628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5afe-1d35-415f-a44b-4e885c6bf9bd None admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:39.557910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5afe-1d35-415f-a44b-4e885c6bf9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:39.558184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f5afe-1d35-415f-a44b-4e885c6bf9bd None admin] PATH_INFO: `/v3/services` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:39.559157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f5afe-1d35-415f-a44b-4e885c6bf9bd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:54:39.559460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f5afe-1d35-415f-a44b-4e885c6bf9bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:39.560866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f5afe-1d35-415f-a44b-4e885c6bf9bd None admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:39.565870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 659/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Tue Jul 5 15:54:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:54:39.571532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed883578-5171-411a-9f28-783e5003fec8 None admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:54:39.670101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed883578-5171-411a-9f28-783e5003fec8 None admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:54:39.707662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d68c1e-2175-47e5-8f79-64f1d285c2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:54:39.708307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d68c1e-2175-47e5-8f79-64f1d285c2f2 None admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:54:39.708585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d68c1e-2175-47e5-8f79-64f1d285c2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:54:39.708885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d68c1e-2175-47e5-8f79-64f1d285c2f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:54:39.709620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d68c1e-2175-47e5-8f79-64f1d285c2f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:54:39.710981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d68c1e-2175-47e5-8f79-64f1d285c2f2 None admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:54:39.792536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 647/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Tue Jul 5 15:54:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:54:49.562929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=30 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:54:49.787856 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:35.893593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73930b3-3e43-46b3-ad23-13e894cfff98 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:35.894626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73930b3-3e43-46b3-ad23-13e894cfff98 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:35.895136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73930b3-3e43-46b3-ad23-13e894cfff98 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:35.903574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7889a4e-9b18-4954-ba2c-f90dee6a7a2d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:35.903958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7889a4e-9b18-4954-ba2c-f90dee6a7a2d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:35.904127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7889a4e-9b18-4954-ba2c-f90dee6a7a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:35.908929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 660/1307] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:00:35 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:00:36.061644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f7889a4e-9b18-4954-ba2c-f90dee6a7a2d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:00:36.063136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f7889a4e-9b18-4954-ba2c-f90dee6a7a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:36.193297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 648/1308] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:00:36.250510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74da90-993b-407f-86b6-df4f4505d76f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:36.250710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74da90-993b-407f-86b6-df4f4505d76f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:36.250977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74da90-993b-407f-86b6-df4f4505d76f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:36.252991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 661/1309] 10.222.0.45 () {54 vars in 964 bytes} [Tue Jul 5 16:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:00:36.259477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fbcdb-4637-4042-952a-5dade86d941d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:36.259776 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fbcdb-4637-4042-952a-5dade86d941d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:36.260056 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fbcdb-4637-4042-952a-5dade86d941d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:36.359349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-556fbcdb-4637-4042-952a-5dade86d941d None None] MFA Rules not processed for user `887d2ad3cbf7459a8a59e7442ba9744e`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:00:36.360963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-556fbcdb-4637-4042-952a-5dade86d941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:36.387902 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:36.448804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 649/1310] 10.222.0.45 () {58 vars in 1061 bytes} [Tue Jul 5 16:00:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:00:36.459684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74da90-993b-407f-86b6-df4f4505d76f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:00:36.461457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6d74da90-993b-407f-86b6-df4f4505d76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:36.585538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74da90-993b-407f-86b6-df4f4505d76f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:00:36.664174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95879e-2902-485f-8047-db6627d0d094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:00:36.664867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95879e-2902-485f-8047-db6627d0d094 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:36.665162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95879e-2902-485f-8047-db6627d0d094 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:36.665440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95879e-2902-485f-8047-db6627d0d094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:36.667099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0b95879e-2902-485f-8047-db6627d0d094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:36.725940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:36.781915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95879e-2902-485f-8047-db6627d0d094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:00:36.783414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95879e-2902-485f-8047-db6627d0d094 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:00:36.803347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 662/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:00:39.117059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702711e-fe60-4d42-bec3-546bf0dae976 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:39.117594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702711e-fe60-4d42-bec3-546bf0dae976 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:39.117707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702711e-fe60-4d42-bec3-546bf0dae976 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:39.118996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 650/1312] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:00:39.124157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00d921-996c-4ccd-8699-4d12354f9bd0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:39.124601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00d921-996c-4ccd-8699-4d12354f9bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:39.124695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00d921-996c-4ccd-8699-4d12354f9bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:39.198923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cf00d921-996c-4ccd-8699-4d12354f9bd0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:00:39.200274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cf00d921-996c-4ccd-8699-4d12354f9bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:39.320558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 663/1313] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:00:39.360369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702711e-fe60-4d42-bec3-546bf0dae976 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:00:39.363058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0702711e-fe60-4d42-bec3-546bf0dae976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:39.469126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702711e-fe60-4d42-bec3-546bf0dae976 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:00:39.550359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:39.553184 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd654f6-46c3-49a0-8a99-282af0ca513e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:00:39.553834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd654f6-46c3-49a0-8a99-282af0ca513e service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:39.554112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd654f6-46c3-49a0-8a99-282af0ca513e service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:39.554380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd654f6-46c3-49a0-8a99-282af0ca513e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:39.555995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-dcd654f6-46c3-49a0-8a99-282af0ca513e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:39.679743 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd654f6-46c3-49a0-8a99-282af0ca513e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:00:39.681295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd654f6-46c3-49a0-8a99-282af0ca513e service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:00:39.700502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 651/1314] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:00:42.173379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247addde-0dc0-4c7d-8f50-c1e66a50f49b None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:42.173910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247addde-0dc0-4c7d-8f50-c1e66a50f49b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:42.173910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247addde-0dc0-4c7d-8f50-c1e66a50f49b None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:42.175381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 664/1315] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:00:42.180619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fee2a3-45ae-4b56-bb28-5fa3a3229a35 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:42.181062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fee2a3-45ae-4b56-bb28-5fa3a3229a35 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:42.181203 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fee2a3-45ae-4b56-bb28-5fa3a3229a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:42.244099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-20fee2a3-45ae-4b56-bb28-5fa3a3229a35 None None] MFA Rules not processed for user `e64e34286da842d2abb6f973aee3a086`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:00:42.245327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-20fee2a3-45ae-4b56-bb28-5fa3a3229a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:42.297208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:42.340199 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 652/1316] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:00:42.379795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247addde-0dc0-4c7d-8f50-c1e66a50f49b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:00:42.381451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-247addde-0dc0-4c7d-8f50-c1e66a50f49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:42.391260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:42.486357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247addde-0dc0-4c7d-8f50-c1e66a50f49b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:00:42.557379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415f3b4-a531-4a68-8791-a562459a9c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:00:42.558018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415f3b4-a531-4a68-8791-a562459a9c93 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:42.558299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415f3b4-a531-4a68-8791-a562459a9c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:42.558572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415f3b4-a531-4a68-8791-a562459a9c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:42.560193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4415f3b4-a531-4a68-8791-a562459a9c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:42.624130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:42.669681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415f3b4-a531-4a68-8791-a562459a9c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:00:42.671119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415f3b4-a531-4a68-8791-a562459a9c93 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:00:42.689671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 665/1317] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:00:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:00:45.482810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d8596-c948-487e-b247-e1aedbf5d7f9 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:45.483364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d8596-c948-487e-b247-e1aedbf5d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:45.483364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d8596-c948-487e-b247-e1aedbf5d7f9 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:45.484733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 653/1318] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:00:45.489942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e0ea5-b330-4b33-a984-63efe15013c8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:45.490331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e0ea5-b330-4b33-a984-63efe15013c8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:45.490502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e0ea5-b330-4b33-a984-63efe15013c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:45.554655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d54e0ea5-b330-4b33-a984-63efe15013c8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:00:45.555771 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d54e0ea5-b330-4b33-a984-63efe15013c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:45.649316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 666/1319] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:00:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:00:45.701886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235701f6-b156-40b6-835c-e8b2fe1e3318 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:45.702171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235701f6-b156-40b6-835c-e8b2fe1e3318 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:45.702449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235701f6-b156-40b6-835c-e8b2fe1e3318 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:45.703804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 654/1320] 10.222.0.45 () {54 vars in 964 bytes} [Tue Jul 5 16:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:00:45.710026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173dd84-38cf-4770-8c3b-3f4bb44cf7d9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:45.710320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173dd84-38cf-4770-8c3b-3f4bb44cf7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:45.710602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173dd84-38cf-4770-8c3b-3f4bb44cf7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:45.731402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:45.785520 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2173dd84-38cf-4770-8c3b-3f4bb44cf7d9 None None] MFA Rules not processed for user `887d2ad3cbf7459a8a59e7442ba9744e`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:00:45.786718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2173dd84-38cf-4770-8c3b-3f4bb44cf7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:45.860988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 667/1321] 10.222.0.45 () {58 vars in 1061 bytes} [Tue Jul 5 16:00:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:00:45.869841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235701f6-b156-40b6-835c-e8b2fe1e3318 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:00:45.871374 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-235701f6-b156-40b6-835c-e8b2fe1e3318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:45.971712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235701f6-b156-40b6-835c-e8b2fe1e3318 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:00:46.042352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afae1b-684c-4039-9053-56d0d9113a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:00:46.042983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afae1b-684c-4039-9053-56d0d9113a56 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:46.043264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afae1b-684c-4039-9053-56d0d9113a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:46.043536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afae1b-684c-4039-9053-56d0d9113a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:46.045173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-41afae1b-684c-4039-9053-56d0d9113a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:46.052252 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:46.151493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afae1b-684c-4039-9053-56d0d9113a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:00:46.152885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afae1b-684c-4039-9053-56d0d9113a56 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:00:46.170055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 655/1322] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:00:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:00:49.703909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21a06-7304-4971-8882-a6b02bbc5705 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:49.705061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21a06-7304-4971-8882-a6b02bbc5705 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:49.705061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a21a06-7304-4971-8882-a6b02bbc5705 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:49.706443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 668/1323] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:00:49.711586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248d57c-6ae9-4111-bfa3-e88c457cf82b None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:49.712019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248d57c-6ae9-4111-bfa3-e88c457cf82b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:49.712282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248d57c-6ae9-4111-bfa3-e88c457cf82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:49.776311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f248d57c-6ae9-4111-bfa3-e88c457cf82b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:00:49.777554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f248d57c-6ae9-4111-bfa3-e88c457cf82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:49.834625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:49.875115 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 656/1324] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:00:49.912199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a21a06-7304-4971-8882-a6b02bbc5705 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:00:49.913882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c5a21a06-7304-4971-8882-a6b02bbc5705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:50.023055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a21a06-7304-4971-8882-a6b02bbc5705 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:00:50.036454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:50.101587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316c05a-54a4-404a-add8-ed510c653fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:00:50.102240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316c05a-54a4-404a-add8-ed510c653fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:50.102523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316c05a-54a4-404a-add8-ed510c653fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:50.102796 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316c05a-54a4-404a-add8-ed510c653fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:50.104462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8316c05a-54a4-404a-add8-ed510c653fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:50.213294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316c05a-54a4-404a-add8-ed510c653fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:00:50.214738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316c05a-54a4-404a-add8-ed510c653fe9 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:00:50.233605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 669/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:00:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:00:52.318264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f6e88-5550-4dfe-8589-6302a37289e0 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:52.318767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f6e88-5550-4dfe-8589-6302a37289e0 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:52.318870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f6e88-5550-4dfe-8589-6302a37289e0 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:52.320186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 657/1326] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:00:52.325536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db17d3-afe1-4d6d-b144-7dc644e8f9ff None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:52.326020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db17d3-afe1-4d6d-b144-7dc644e8f9ff None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:52.326227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db17d3-afe1-4d6d-b144-7dc644e8f9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:52.374512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:52.394288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b6db17d3-afe1-4d6d-b144-7dc644e8f9ff None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:00:52.395524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b6db17d3-afe1-4d6d-b144-7dc644e8f9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:52.496709 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 670/1327] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:00:52.532034 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f6e88-5550-4dfe-8589-6302a37289e0 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:00:52.533682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f76f6e88-5550-4dfe-8589-6302a37289e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:52.639458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f6e88-5550-4dfe-8589-6302a37289e0 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:00:52.711800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b437b3-cf6a-4636-96d8-8aa0fb663315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:00:52.712446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b437b3-cf6a-4636-96d8-8aa0fb663315 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:52.712731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b437b3-cf6a-4636-96d8-8aa0fb663315 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:52.713039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b437b3-cf6a-4636-96d8-8aa0fb663315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:52.714691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d6b437b3-cf6a-4636-96d8-8aa0fb663315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:52.732586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:52.829794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b437b3-cf6a-4636-96d8-8aa0fb663315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:00:52.831351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b437b3-cf6a-4636-96d8-8aa0fb663315 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:00:52.849685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 658/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:00:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:00:54.921904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebe650-a884-46a8-9888-3d569b66bbb9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:54.922461 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebe650-a884-46a8-9888-3d569b66bbb9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:54.922461 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebe650-a884-46a8-9888-3d569b66bbb9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:54.924367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 671/1329] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:00:54.929776 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc7900-7df2-4273-b00b-45d73c0388ae None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:54.930182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc7900-7df2-4273-b00b-45d73c0388ae None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:54.930341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc7900-7df2-4273-b00b-45d73c0388ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:54.994639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e1fc7900-7df2-4273-b00b-45d73c0388ae None None] MFA Rules not processed for user `e64e34286da842d2abb6f973aee3a086`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:00:54.995855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e1fc7900-7df2-4273-b00b-45d73c0388ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:55.062159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:55.095914 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 659/1330] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:00:55.134288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebe650-a884-46a8-9888-3d569b66bbb9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:00:55.135955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2cebe650-a884-46a8-9888-3d569b66bbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:55.238195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebe650-a884-46a8-9888-3d569b66bbb9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:00:55.257609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:55.314051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66333b21-1d2b-4563-a8d2-277429c5c734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:00:55.314703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333b21-1d2b-4563-a8d2-277429c5c734 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:55.314984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333b21-1d2b-4563-a8d2-277429c5c734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:55.315258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333b21-1d2b-4563-a8d2-277429c5c734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:55.316960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-66333b21-1d2b-4563-a8d2-277429c5c734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:55.427353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66333b21-1d2b-4563-a8d2-277429c5c734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:00:55.428869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66333b21-1d2b-4563-a8d2-277429c5c734 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:00:55.446976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 672/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:00:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:00:57.926830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6310c-1e24-42ab-9205-7ddc10fe2fa1 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:57.927382 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6310c-1e24-42ab-9205-7ddc10fe2fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:57.927382 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6310c-1e24-42ab-9205-7ddc10fe2fa1 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:57.928831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 660/1332] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:00:57.934607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57ed94-409f-4552-8e3b-36781c787144 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:57.935063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57ed94-409f-4552-8e3b-36781c787144 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:57.935162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57ed94-409f-4552-8e3b-36781c787144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:57.999527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:58.003174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2d57ed94-409f-4552-8e3b-36781c787144 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:00:58.004418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2d57ed94-409f-4552-8e3b-36781c787144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:58.117097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 673/1333] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:00:58.158655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6310c-1e24-42ab-9205-7ddc10fe2fa1 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:00:58.160205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-dcc6310c-1e24-42ab-9205-7ddc10fe2fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:58.266063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6310c-1e24-42ab-9205-7ddc10fe2fa1 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:00:58.337360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851685a7-e7c3-45f8-9bfa-816baf892967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:00:58.338001 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851685a7-e7c3-45f8-9bfa-816baf892967 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:00:58.338280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851685a7-e7c3-45f8-9bfa-816baf892967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:00:58.338550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851685a7-e7c3-45f8-9bfa-816baf892967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:00:58.340169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-851685a7-e7c3-45f8-9bfa-816baf892967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:00:58.367834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:00:58.448140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851685a7-e7c3-45f8-9bfa-816baf892967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:00:58.449597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851685a7-e7c3-45f8-9bfa-816baf892967 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:00:58.467667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 661/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:00.515192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd020f-be93-4420-a9a3-57b2b35483f9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:00.515764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd020f-be93-4420-a9a3-57b2b35483f9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:00.515764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd020f-be93-4420-a9a3-57b2b35483f9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:00.517181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 674/1335] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:00.522346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c4dee-c0ef-4beb-af7f-c11b4fa01899 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:00.522759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c4dee-c0ef-4beb-af7f-c11b4fa01899 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:00.522939 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c4dee-c0ef-4beb-af7f-c11b4fa01899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:00.586785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-981c4dee-c0ef-4beb-af7f-c11b4fa01899 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:00.587929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-981c4dee-c0ef-4beb-af7f-c11b4fa01899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:00.667745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:01:00.690124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 662/1336] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:00.729766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd020f-be93-4420-a9a3-57b2b35483f9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:00.731286 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-17fd020f-be93-4420-a9a3-57b2b35483f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:00.832404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd020f-be93-4420-a9a3-57b2b35483f9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:00.860511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:01:00.904709 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a6b62-c99c-40c8-8c47-1a8f2b0b3745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:00.905379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a6b62-c99c-40c8-8c47-1a8f2b0b3745 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:00.905665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a6b62-c99c-40c8-8c47-1a8f2b0b3745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:00.905943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a6b62-c99c-40c8-8c47-1a8f2b0b3745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:00.908252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-717a6b62-c99c-40c8-8c47-1a8f2b0b3745 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:01.012879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a6b62-c99c-40c8-8c47-1a8f2b0b3745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:01.014316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a6b62-c99c-40c8-8c47-1a8f2b0b3745 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:01.033263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 675/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:03.767485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb798c-c125-4539-988c-7162d76573fd None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:03.768041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb798c-c125-4539-988c-7162d76573fd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:03.768041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb798c-c125-4539-988c-7162d76573fd None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:03.769445 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 663/1338] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:03.774731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf4a13-037f-497c-922d-fa44429a1cb9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:03.775118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf4a13-037f-497c-922d-fa44429a1cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:03.775292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf4a13-037f-497c-922d-fa44429a1cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:03.840734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9baf4a13-037f-497c-922d-fa44429a1cb9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:03.841966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9baf4a13-037f-497c-922d-fa44429a1cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:03.850720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:01:03.938442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 676/1339] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:03.975454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb798c-c125-4539-988c-7162d76573fd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:03.977033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-decb798c-c125-4539-988c-7162d76573fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:04.080404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb798c-c125-4539-988c-7162d76573fd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:04.151614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb6a0-63df-4084-aa3d-70a8875d7601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:04.152231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb6a0-63df-4084-aa3d-70a8875d7601 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:04.152513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb6a0-63df-4084-aa3d-70a8875d7601 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:04.152804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb6a0-63df-4084-aa3d-70a8875d7601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:04.154423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-bc1cb6a0-63df-4084-aa3d-70a8875d7601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:04.188814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:01:04.259717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cb6a0-63df-4084-aa3d-70a8875d7601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:04.261168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cb6a0-63df-4084-aa3d-70a8875d7601 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:04.279738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 664/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:08.489741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121a220-c289-4499-8d4b-786039231691 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:08.490290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121a220-c289-4499-8d4b-786039231691 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:08.490290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121a220-c289-4499-8d4b-786039231691 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:08.491822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 677/1341] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:08.496974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62d542-63b4-4546-9824-8ba9e33de153 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:08.497426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62d542-63b4-4546-9824-8ba9e33de153 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:08.497530 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62d542-63b4-4546-9824-8ba9e33de153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:08.573585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-7e62d542-63b4-4546-9824-8ba9e33de153 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:08.574906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7e62d542-63b4-4546-9824-8ba9e33de153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:08.684371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:01:08.697275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 665/1342] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:08.737133 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121a220-c289-4499-8d4b-786039231691 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:08.739294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5121a220-c289-4499-8d4b-786039231691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:08.835711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121a220-c289-4499-8d4b-786039231691 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:08.839247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49555f2c-65e6-4377-9740-5fa3f2f9da50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:08.839890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49555f2c-65e6-4377-9740-5fa3f2f9da50 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:08.840173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49555f2c-65e6-4377-9740-5fa3f2f9da50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:08.840536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49555f2c-65e6-4377-9740-5fa3f2f9da50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:08.850252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49555f2c-65e6-4377-9740-5fa3f2f9da50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:08.851688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49555f2c-65e6-4377-9740-5fa3f2f9da50 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:08.862173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 678/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:11.841285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8f388-fcb2-4814-9186-40085e177f77 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:11.841821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8f388-fcb2-4814-9186-40085e177f77 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:11.841912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8f388-fcb2-4814-9186-40085e177f77 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:11.843402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 666/1344] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:11.849035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82aa435-2243-4a7c-abfd-d05f15b1c092 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:11.849424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82aa435-2243-4a7c-abfd-d05f15b1c092 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:11.849587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82aa435-2243-4a7c-abfd-d05f15b1c092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:11.879052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e82aa435-2243-4a7c-abfd-d05f15b1c092 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:11.880300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e82aa435-2243-4a7c-abfd-d05f15b1c092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:11.911014 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:01:11.960258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 679/1345] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:11.997528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8f388-fcb2-4814-9186-40085e177f77 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:12.008610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8f388-fcb2-4814-9186-40085e177f77 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:12.012062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f23694-01ad-4f72-9335-81c000a9acac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:12.012696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f23694-01ad-4f72-9335-81c000a9acac service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:12.013018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f23694-01ad-4f72-9335-81c000a9acac service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:12.013296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f23694-01ad-4f72-9335-81c000a9acac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:12.022229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f23694-01ad-4f72-9335-81c000a9acac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:12.023657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f23694-01ad-4f72-9335-81c000a9acac service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:12.033349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 667/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:16.323648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074a580-7604-4c5e-8df6-5de4739439ab None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:16.324153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074a580-7604-4c5e-8df6-5de4739439ab None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:16.324272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074a580-7604-4c5e-8df6-5de4739439ab None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:16.325641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 680/1347] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:16.332884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac25fb5-c112-424f-a2f7-2e27064ee62f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:16.332884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac25fb5-c112-424f-a2f7-2e27064ee62f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:16.332884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac25fb5-c112-424f-a2f7-2e27064ee62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:16.349577 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-2ac25fb5-c112-424f-a2f7-2e27064ee62f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:16.350940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2ac25fb5-c112-424f-a2f7-2e27064ee62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:16.359073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 668/1348] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:16.396986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074a580-7604-4c5e-8df6-5de4739439ab None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:16.407534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074a580-7604-4c5e-8df6-5de4739439ab None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:16.411057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d02b03-2e72-484e-bdd0-5d69dc1e6b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:16.411676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d02b03-2e72-484e-bdd0-5d69dc1e6b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:16.411958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d02b03-2e72-484e-bdd0-5d69dc1e6b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:16.412230 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d02b03-2e72-484e-bdd0-5d69dc1e6b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:16.421036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d02b03-2e72-484e-bdd0-5d69dc1e6b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:16.422405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d02b03-2e72-484e-bdd0-5d69dc1e6b7d service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:16.432103 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 681/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:21.475320 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86e475-9b3a-4501-8760-35f222890aea None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:21.476320 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86e475-9b3a-4501-8760-35f222890aea None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:21.476817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86e475-9b3a-4501-8760-35f222890aea None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:21.478686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 669/1350] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:21.491310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ef33b-5d68-47f2-8cac-df7765050e88 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:21.493244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ef33b-5d68-47f2-8cac-df7765050e88 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:21.493876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ef33b-5d68-47f2-8cac-df7765050e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:21.581756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ab8ef33b-5d68-47f2-8cac-df7765050e88 None None] MFA Rules not processed for user `75b7f541993b4a7c915c9e6d440e2674`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:21.583212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ab8ef33b-5d68-47f2-8cac-df7765050e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:21.660675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 682/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:01:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:01:25.798262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe665c98-b1df-40c5-aa42-80b57da66194 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:25.798796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe665c98-b1df-40c5-aa42-80b57da66194 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:25.799051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe665c98-b1df-40c5-aa42-80b57da66194 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:25.800631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 670/1352] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:25.805357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136fbc5-4fe8-446f-900d-f940cf24589f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:25.806243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136fbc5-4fe8-446f-900d-f940cf24589f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:25.806731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136fbc5-4fe8-446f-900d-f940cf24589f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:25.825241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d136fbc5-4fe8-446f-900d-f940cf24589f None None] MFA Rules not processed for user `75b7f541993b4a7c915c9e6d440e2674`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:25.826428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d136fbc5-4fe8-446f-900d-f940cf24589f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:25.835587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 683/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:01:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:26.345880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=13 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:01:27.339603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c672733-4ee7-48a6-b23d-0ff07408298e None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:27.340219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c672733-4ee7-48a6-b23d-0ff07408298e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:27.340219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c672733-4ee7-48a6-b23d-0ff07408298e None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:27.341510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 671/1354] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:27.347060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6484e9-6e00-46b0-8db5-cf5ca7a338e8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:27.347479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6484e9-6e00-46b0-8db5-cf5ca7a338e8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:27.347616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6484e9-6e00-46b0-8db5-cf5ca7a338e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:27.366272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3a6484e9-6e00-46b0-8db5-cf5ca7a338e8 None None] MFA Rules not processed for user `75b7f541993b4a7c915c9e6d440e2674`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:27.367457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3a6484e9-6e00-46b0-8db5-cf5ca7a338e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:27.376602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 684/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:01:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:30.871609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50f70-ab23-4929-8103-d497fca53b63 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:30.872127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50f70-ab23-4929-8103-d497fca53b63 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:30.872127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50f70-ab23-4929-8103-d497fca53b63 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:30.879190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a882c-453f-4401-9c0c-7509b31b3a86 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:30.880444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a882c-453f-4401-9c0c-7509b31b3a86 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:30.881092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a882c-453f-4401-9c0c-7509b31b3a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:30.883014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 672/1356] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:30.903050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-539a882c-453f-4401-9c0c-7509b31b3a86 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:30.904493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-539a882c-453f-4401-9c0c-7509b31b3a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:30.913926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 685/1357] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:30.970404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ba2f4-a9a6-4594-9fa4-b0e363d3730d None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:30.970831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ba2f4-a9a6-4594-9fa4-b0e363d3730d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:30.971392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ba2f4-a9a6-4594-9fa4-b0e363d3730d None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:30.974323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 673/1358] 10.222.0.45 () {54 vars in 961 bytes} [Tue Jul 5 16:01:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:30.980834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229331fa-814c-4cfb-ab8a-5923d491efda None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:30.981134 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229331fa-814c-4cfb-ab8a-5923d491efda None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:30.981419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229331fa-814c-4cfb-ab8a-5923d491efda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:31.034064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:01:31.041944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-229331fa-814c-4cfb-ab8a-5923d491efda None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:31.043151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-229331fa-814c-4cfb-ab8a-5923d491efda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:31.143240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 686/1359] 10.222.0.45 () {58 vars in 1058 bytes} [Tue Jul 5 16:01:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:31.151359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ba2f4-a9a6-4594-9fa4-b0e363d3730d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:31.168875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ba2f4-a9a6-4594-9fa4-b0e363d3730d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:31.174668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897938ae-d1ff-42eb-9512-ca81b1c16003 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:31.175412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897938ae-d1ff-42eb-9512-ca81b1c16003 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:31.175818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897938ae-d1ff-42eb-9512-ca81b1c16003 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:31.176191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897938ae-d1ff-42eb-9512-ca81b1c16003 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:31.185338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897938ae-d1ff-42eb-9512-ca81b1c16003 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:31.187613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897938ae-d1ff-42eb-9512-ca81b1c16003 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:31.197910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 674/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:33.418084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073f431-b501-438b-b7a8-117fe074cc02 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:33.418639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073f431-b501-438b-b7a8-117fe074cc02 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:33.418639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073f431-b501-438b-b7a8-117fe074cc02 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:33.420007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 687/1361] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:33.425652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b95b9-1478-40c8-ad4b-f4700d59c244 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:33.426070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b95b9-1478-40c8-ad4b-f4700d59c244 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:33.426223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b95b9-1478-40c8-ad4b-f4700d59c244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:33.444528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4f0b95b9-1478-40c8-ad4b-f4700d59c244 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:33.445849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4f0b95b9-1478-40c8-ad4b-f4700d59c244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:33.454200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 675/1362] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:33.501127 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073f431-b501-438b-b7a8-117fe074cc02 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:33.516803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073f431-b501-438b-b7a8-117fe074cc02 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:33.522596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd1104-5558-4126-9dfe-0ee268bef914 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:33.523360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd1104-5558-4126-9dfe-0ee268bef914 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:33.523653 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd1104-5558-4126-9dfe-0ee268bef914 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:33.523935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd1104-5558-4126-9dfe-0ee268bef914 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:33.533028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd1104-5558-4126-9dfe-0ee268bef914 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:33.534661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd1104-5558-4126-9dfe-0ee268bef914 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:33.544647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 688/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:36.062089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc4cde-f0cf-43bb-992d-5a3bb91640ce None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:36.062586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc4cde-f0cf-43bb-992d-5a3bb91640ce None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:36.062667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc4cde-f0cf-43bb-992d-5a3bb91640ce None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:36.064249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 676/1364] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:36.069435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcb91-5a9a-4331-b1c2-75bd717c767f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:36.069813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcb91-5a9a-4331-b1c2-75bd717c767f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:36.069933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcb91-5a9a-4331-b1c2-75bd717c767f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:36.088329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b71fcb91-5a9a-4331-b1c2-75bd717c767f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:36.089562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b71fcb91-5a9a-4331-b1c2-75bd717c767f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:36.097888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 689/1365] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:36.144875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de51e0-7d2c-4352-a05a-e2d3005ade85 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:36.145235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de51e0-7d2c-4352-a05a-e2d3005ade85 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:36.145581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de51e0-7d2c-4352-a05a-e2d3005ade85 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:36.147029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 677/1366] 10.222.0.45 () {54 vars in 961 bytes} [Tue Jul 5 16:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:36.152418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15093d1-3bb1-437e-a555-6935593b5465 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:36.156002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15093d1-3bb1-437e-a555-6935593b5465 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:36.156002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15093d1-3bb1-437e-a555-6935593b5465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:36.171382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b15093d1-3bb1-437e-a555-6935593b5465 None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:36.172557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b15093d1-3bb1-437e-a555-6935593b5465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:36.182053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 690/1367] 10.222.0.45 () {58 vars in 1058 bytes} [Tue Jul 5 16:01:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:36.189793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de51e0-7d2c-4352-a05a-e2d3005ade85 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:36.205138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de51e0-7d2c-4352-a05a-e2d3005ade85 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:36.210795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31519368-64f3-4435-a931-88b86316f83b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:36.211447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31519368-64f3-4435-a931-88b86316f83b service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:36.211730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31519368-64f3-4435-a931-88b86316f83b service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:36.212007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31519368-64f3-4435-a931-88b86316f83b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:36.221326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31519368-64f3-4435-a931-88b86316f83b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:36.222914 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31519368-64f3-4435-a931-88b86316f83b service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:36.233074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 678/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:01:38.470631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e85bf-7d6b-4bf7-9caf-e788deada2b7 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:38.471162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e85bf-7d6b-4bf7-9caf-e788deada2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:38.471162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e85bf-7d6b-4bf7-9caf-e788deada2b7 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:38.472566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 691/1369] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:38.477962 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1199fed-8809-4be8-9c0f-10b93a7907ae None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:38.481272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1199fed-8809-4be8-9c0f-10b93a7907ae None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:38.481272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1199fed-8809-4be8-9c0f-10b93a7907ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:38.496871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e1199fed-8809-4be8-9c0f-10b93a7907ae None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:38.498286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e1199fed-8809-4be8-9c0f-10b93a7907ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:38.506324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 679/1370] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:38.552368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e85bf-7d6b-4bf7-9caf-e788deada2b7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:38.566514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e85bf-7d6b-4bf7-9caf-e788deada2b7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:38.572044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42aa152-d276-4dd5-a620-d17911d22572 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:38.572683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aa152-d276-4dd5-a620-d17911d22572 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:38.573014 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aa152-d276-4dd5-a620-d17911d22572 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:38.573289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aa152-d276-4dd5-a620-d17911d22572 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:38.582220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42aa152-d276-4dd5-a620-d17911d22572 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:38.583830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42aa152-d276-4dd5-a620-d17911d22572 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:38.594251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 692/1371] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:38.984322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914703bd-9628-4ce3-a4b4-bc01f45c1b27 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:38.984929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914703bd-9628-4ce3-a4b4-bc01f45c1b27 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:38.985381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914703bd-9628-4ce3-a4b4-bc01f45c1b27 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:38.986887 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 680/1372] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:38.992423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4692c-2196-4f91-882e-04a6f5833267 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:38.992967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4692c-2196-4f91-882e-04a6f5833267 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:38.993426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4692c-2196-4f91-882e-04a6f5833267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:39.013321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f5f4692c-2196-4f91-882e-04a6f5833267 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:39.014849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f5f4692c-2196-4f91-882e-04a6f5833267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:39.023544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 693/1373] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:01:39.330513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0999cdf-e771-43a1-b09d-f98c97620adc None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:39.330804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0999cdf-e771-43a1-b09d-f98c97620adc None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:39.331295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0999cdf-e771-43a1-b09d-f98c97620adc None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:39.332709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 681/1374] 10.222.0.45 () {54 vars in 963 bytes} [Tue Jul 5 16:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:39.338244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726a897-62f1-45a0-b755-1e9e12807593 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:39.338591 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726a897-62f1-45a0-b755-1e9e12807593 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:39.347333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726a897-62f1-45a0-b755-1e9e12807593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:39.391570 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1726a897-62f1-45a0-b755-1e9e12807593 None None] MFA Rules not processed for user `805a9dd2d0934359b7ca5fe629b96d38`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:39.392792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1726a897-62f1-45a0-b755-1e9e12807593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:39.456754 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 694/1375] 10.222.0.45 () {58 vars in 1060 bytes} [Tue Jul 5 16:01:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:39.464050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0999cdf-e771-43a1-b09d-f98c97620adc None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:39.476393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0999cdf-e771-43a1-b09d-f98c97620adc None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:39.480687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79a6a2-66d1-4984-b211-810d9c94447b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:39.481444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79a6a2-66d1-4984-b211-810d9c94447b service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:39.481801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79a6a2-66d1-4984-b211-810d9c94447b service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:39.482434 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79a6a2-66d1-4984-b211-810d9c94447b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:39.494926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79a6a2-66d1-4984-b211-810d9c94447b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:39.496543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79a6a2-66d1-4984-b211-810d9c94447b service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:39.508460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 682/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:01:40.774049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af1931-2006-4cd2-84d6-a00035180b38 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:40.774591 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af1931-2006-4cd2-84d6-a00035180b38 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:40.774591 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af1931-2006-4cd2-84d6-a00035180b38 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:40.776146 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 695/1377] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:40.782081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6469-aae3-460a-999c-3081eee22bdb None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:40.782510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6469-aae3-460a-999c-3081eee22bdb None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:40.782763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6469-aae3-460a-999c-3081eee22bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:40.800257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-916c6469-aae3-460a-999c-3081eee22bdb None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:40.801484 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-916c6469-aae3-460a-999c-3081eee22bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:40.809249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 683/1378] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:40.859569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af1931-2006-4cd2-84d6-a00035180b38 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:40.874581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af1931-2006-4cd2-84d6-a00035180b38 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:40.879904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6123521-473d-4ba4-a641-b079c603140b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:40.880538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6123521-473d-4ba4-a641-b079c603140b service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:40.880845 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6123521-473d-4ba4-a641-b079c603140b service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:40.881119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6123521-473d-4ba4-a641-b079c603140b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:40.890072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6123521-473d-4ba4-a641-b079c603140b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:40.891659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6123521-473d-4ba4-a641-b079c603140b service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:40.895637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:01:40.901566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 696/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:43.123113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fdca3-60d1-482e-bb29-9685d1a61c24 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:43.123631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fdca3-60d1-482e-bb29-9685d1a61c24 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:43.124288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fdca3-60d1-482e-bb29-9685d1a61c24 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:43.125816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 684/1380] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:43.132162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200acc55-c5ad-4793-a8ae-c69bd911a0de None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:43.133015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200acc55-c5ad-4793-a8ae-c69bd911a0de None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:43.133452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200acc55-c5ad-4793-a8ae-c69bd911a0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:43.155088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-200acc55-c5ad-4793-a8ae-c69bd911a0de None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:43.157929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-200acc55-c5ad-4793-a8ae-c69bd911a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:43.168334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 697/1381] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:43.217091 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fdca3-60d1-482e-bb29-9685d1a61c24 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:43.234325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fdca3-60d1-482e-bb29-9685d1a61c24 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:43.240014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fe0a2-017c-4fd8-9fa0-80ba69b58178 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:43.240654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fe0a2-017c-4fd8-9fa0-80ba69b58178 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:43.240972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fe0a2-017c-4fd8-9fa0-80ba69b58178 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:43.241254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fe0a2-017c-4fd8-9fa0-80ba69b58178 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:43.250683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fe0a2-017c-4fd8-9fa0-80ba69b58178 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:43.252292 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fe0a2-017c-4fd8-9fa0-80ba69b58178 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:43.261897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 685/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:45.461320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363dbd0-2793-4123-bdfb-c7793f678a30 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:45.461852 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363dbd0-2793-4123-bdfb-c7793f678a30 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:45.461852 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363dbd0-2793-4123-bdfb-c7793f678a30 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:45.463239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 698/1383] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:45.468415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589fe27-24a1-48de-85d8-4f3b26ecae63 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:45.468878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589fe27-24a1-48de-85d8-4f3b26ecae63 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:45.469016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589fe27-24a1-48de-85d8-4f3b26ecae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:45.490737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f589fe27-24a1-48de-85d8-4f3b26ecae63 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:45.492155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f589fe27-24a1-48de-85d8-4f3b26ecae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:45.513055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 686/1384] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:45.542181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363dbd0-2793-4123-bdfb-c7793f678a30 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:45.558554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363dbd0-2793-4123-bdfb-c7793f678a30 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:45.564503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1d0b7-4fb2-4e49-9828-02b47d607d31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:45.565182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1d0b7-4fb2-4e49-9828-02b47d607d31 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:45.565494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1d0b7-4fb2-4e49-9828-02b47d607d31 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:45.565776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1d0b7-4fb2-4e49-9828-02b47d607d31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:45.574343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1d0b7-4fb2-4e49-9828-02b47d607d31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:45.575923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1d0b7-4fb2-4e49-9828-02b47d607d31 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:45.586076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 699/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:47.709145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70fcf6-5448-470d-aa89-8ddf2df2a18c None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:47.709661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70fcf6-5448-470d-aa89-8ddf2df2a18c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:47.709755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70fcf6-5448-470d-aa89-8ddf2df2a18c None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:47.711182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 687/1386] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:47.716597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58e579-9c01-4276-9231-dd16746eecd8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:47.717162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58e579-9c01-4276-9231-dd16746eecd8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:47.717786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58e579-9c01-4276-9231-dd16746eecd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:47.737017 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3c58e579-9c01-4276-9231-dd16746eecd8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:47.738300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3c58e579-9c01-4276-9231-dd16746eecd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:47.746638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 700/1387] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:47.792375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70fcf6-5448-470d-aa89-8ddf2df2a18c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:47.808343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70fcf6-5448-470d-aa89-8ddf2df2a18c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:47.814040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae28634-a2d5-42d9-b9b5-cfa3b5a091e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:47.814677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae28634-a2d5-42d9-b9b5-cfa3b5a091e6 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:47.814962 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae28634-a2d5-42d9-b9b5-cfa3b5a091e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:47.815238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae28634-a2d5-42d9-b9b5-cfa3b5a091e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:47.823834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae28634-a2d5-42d9-b9b5-cfa3b5a091e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:47.825456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae28634-a2d5-42d9-b9b5-cfa3b5a091e6 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:47.835109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 688/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:50.114695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d486aa-ebd2-431f-9233-48c69edd17df None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:50.115234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d486aa-ebd2-431f-9233-48c69edd17df None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:50.115415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d486aa-ebd2-431f-9233-48c69edd17df None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:50.117221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 701/1389] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:50.122338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae128a0-69d2-4081-b812-570026c57bd6 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:50.122935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae128a0-69d2-4081-b812-570026c57bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:50.122935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae128a0-69d2-4081-b812-570026c57bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:50.141988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-dae128a0-69d2-4081-b812-570026c57bd6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:50.143530 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-dae128a0-69d2-4081-b812-570026c57bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:50.153387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 689/1390] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:50.200726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d486aa-ebd2-431f-9233-48c69edd17df None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:50.216524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d486aa-ebd2-431f-9233-48c69edd17df None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:50.222936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf35cd-f307-473c-bf69-7805411af0f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:50.223821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf35cd-f307-473c-bf69-7805411af0f2 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:50.224173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf35cd-f307-473c-bf69-7805411af0f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:50.224522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf35cd-f307-473c-bf69-7805411af0f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:50.236319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf35cd-f307-473c-bf69-7805411af0f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:50.238378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf35cd-f307-473c-bf69-7805411af0f2 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:50.251198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 702/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:52.630714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202d6c-7717-4fdd-8a57-095244e493c2 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:52.631387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202d6c-7717-4fdd-8a57-095244e493c2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:52.631387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202d6c-7717-4fdd-8a57-095244e493c2 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:52.633318 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 690/1392] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:52.640336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47603ee7-792e-4cc0-8698-b80814a9efc7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:52.640823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47603ee7-792e-4cc0-8698-b80814a9efc7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:52.641061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47603ee7-792e-4cc0-8698-b80814a9efc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:52.662795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-47603ee7-792e-4cc0-8698-b80814a9efc7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:52.664293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-47603ee7-792e-4cc0-8698-b80814a9efc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:52.675083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 703/1393] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:52.730343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202d6c-7717-4fdd-8a57-095244e493c2 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:52.745647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202d6c-7717-4fdd-8a57-095244e493c2 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:52.751650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb6f8c-5041-47fe-a7f5-2fa1860ff8f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:52.752280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb6f8c-5041-47fe-a7f5-2fa1860ff8f5 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:52.752571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb6f8c-5041-47fe-a7f5-2fa1860ff8f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:52.752863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb6f8c-5041-47fe-a7f5-2fa1860ff8f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:52.761288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb6f8c-5041-47fe-a7f5-2fa1860ff8f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:52.762859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb6f8c-5041-47fe-a7f5-2fa1860ff8f5 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:52.772405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 691/1394] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:01:54.931541 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01545922-8fec-4ce3-8369-15d720625fa1 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:54.932102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01545922-8fec-4ce3-8369-15d720625fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:54.932102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01545922-8fec-4ce3-8369-15d720625fa1 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:54.933507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 704/1395] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:54.938639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437072d-402b-482a-8506-e68a22df3192 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:54.939034 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437072d-402b-482a-8506-e68a22df3192 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:54.939221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437072d-402b-482a-8506-e68a22df3192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:54.957150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-7437072d-402b-482a-8506-e68a22df3192 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:54.958338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7437072d-402b-482a-8506-e68a22df3192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:54.966050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 692/1396] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:55.009337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01545922-8fec-4ce3-8369-15d720625fa1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:55.024293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01545922-8fec-4ce3-8369-15d720625fa1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:55.030306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e4bdb-042c-4a4c-b1f4-2ec7e8b44ea1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:55.030951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e4bdb-042c-4a4c-b1f4-2ec7e8b44ea1 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:55.031236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e4bdb-042c-4a4c-b1f4-2ec7e8b44ea1 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:55.031515 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e4bdb-042c-4a4c-b1f4-2ec7e8b44ea1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:55.039975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e4bdb-042c-4a4c-b1f4-2ec7e8b44ea1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:55.041620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e4bdb-042c-4a4c-b1f4-2ec7e8b44ea1 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:55.042540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36703d71-2bc9-4804-ab4f-1c76a459e8f2 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:55.042834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36703d71-2bc9-4804-ab4f-1c76a459e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:55.043116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36703d71-2bc9-4804-ab4f-1c76a459e8f2 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:55.044475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 693/1397] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 16:01:55.049522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e23e9-4552-4f2e-8b0a-f4a939de2493 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:55.049815 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e23e9-4552-4f2e-8b0a-f4a939de2493 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:55.050096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e23e9-4552-4f2e-8b0a-f4a939de2493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:55.051752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 705/1398] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:55.067738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-9d9e23e9-4552-4f2e-8b0a-f4a939de2493 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:55.068969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9d9e23e9-4552-4f2e-8b0a-f4a939de2493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:55.077629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 694/1399] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:01:55.134019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39ab91-4a9b-4d66-bdf1-74c3c2d8f557 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:55.134316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39ab91-4a9b-4d66-bdf1-74c3c2d8f557 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:55.134600 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39ab91-4a9b-4d66-bdf1-74c3c2d8f557 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:55.135985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 706/1400] 10.222.0.45 () {54 vars in 962 bytes} [Tue Jul 5 16:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:55.141555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a60a4-f7f9-48f1-bf3f-f31880e817a6 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:55.141969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a60a4-f7f9-48f1-bf3f-f31880e817a6 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:55.142308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a60a4-f7f9-48f1-bf3f-f31880e817a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:55.190998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-920a60a4-f7f9-48f1-bf3f-f31880e817a6 None None] MFA Rules not processed for user `a5b712bd7d8d4930a3bfc99581be8cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:55.192125 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-920a60a4-f7f9-48f1-bf3f-f31880e817a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:55.256696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 695/1401] 10.222.0.45 () {58 vars in 1059 bytes} [Tue Jul 5 16:01:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:55.265380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39ab91-4a9b-4d66-bdf1-74c3c2d8f557 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:55.279045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39ab91-4a9b-4d66-bdf1-74c3c2d8f557 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:55.283037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7790d4-cdfe-4373-8bf9-e81866a7c80b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b712bd7d8d4930a3bfc99581be8cb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:55.283853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7790d4-cdfe-4373-8bf9-e81866a7c80b service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:55.284142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7790d4-cdfe-4373-8bf9-e81866a7c80b service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:55.284424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7790d4-cdfe-4373-8bf9-e81866a7c80b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:55.298729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7790d4-cdfe-4373-8bf9-e81866a7c80b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:55.300193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7790d4-cdfe-4373-8bf9-e81866a7c80b service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:55.311212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 707/1402] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:01:56.166964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3895e8d-f4e2-419f-a801-24bcfb8f27b3 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:56.167418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3895e8d-f4e2-419f-a801-24bcfb8f27b3 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:56.167520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3895e8d-f4e2-419f-a801-24bcfb8f27b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:56.185212 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:01:56.332954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-a3895e8d-f4e2-419f-a801-24bcfb8f27b3 None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:56.334156 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a3895e8d-f4e2-419f-a801-24bcfb8f27b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:56.399178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 696/1403] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:01:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:56.444979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74553169-feaa-4220-9190-a3fe07f5d269 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:56.445420 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74553169-feaa-4220-9190-a3fe07f5d269 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:56.445532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74553169-feaa-4220-9190-a3fe07f5d269 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:56.447082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 708/1404] 10.222.0.45 () {54 vars in 942 bytes} [Tue Jul 5 16:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:56.453724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf20108-9943-42ae-b737-6033c6b04b5e None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:56.454076 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf20108-9943-42ae-b737-6033c6b04b5e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:56.454418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf20108-9943-42ae-b737-6033c6b04b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:56.502386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3bf20108-9943-42ae-b737-6033c6b04b5e None None] MFA Rules not processed for user `8da9cdd70af646928ee0edcadbcb524f`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:56.503708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3bf20108-9943-42ae-b737-6033c6b04b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:56.574910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 697/1405] 10.222.0.45 () {58 vars in 1039 bytes} [Tue Jul 5 16:01:56 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:56.584557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74553169-feaa-4220-9190-a3fe07f5d269 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:56.598483 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74553169-feaa-4220-9190-a3fe07f5d269 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:56.603417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f149dc-f627-407d-b7c8-1f4a5da37ffa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:56.604045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f149dc-f627-407d-b7c8-1f4a5da37ffa service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:56.604333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f149dc-f627-407d-b7c8-1f4a5da37ffa service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:56.604610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f149dc-f627-407d-b7c8-1f4a5da37ffa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:56.612843 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f149dc-f627-407d-b7c8-1f4a5da37ffa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:56.614425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f149dc-f627-407d-b7c8-1f4a5da37ffa service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:56.622689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 709/1406] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:01:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:01:57.217977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee603-781e-4e83-a292-58e482b1297d None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:57.218502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee603-781e-4e83-a292-58e482b1297d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:57.218502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee603-781e-4e83-a292-58e482b1297d None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:57.219868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 698/1407] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:01:57.225387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca347d-0af4-4d3b-9b9f-db594ab95650 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:57.226040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca347d-0af4-4d3b-9b9f-db594ab95650 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:57.226538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca347d-0af4-4d3b-9b9f-db594ab95650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:57.245583 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-72ca347d-0af4-4d3b-9b9f-db594ab95650 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:01:57.246927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-72ca347d-0af4-4d3b-9b9f-db594ab95650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:01:57.273899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 710/1408] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:01:57.319781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee603-781e-4e83-a292-58e482b1297d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:01:57.337877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee603-781e-4e83-a292-58e482b1297d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:01:57.345427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158045f7-0270-41bf-8923-e3d2e61d6a3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:01:57.346173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158045f7-0270-41bf-8923-e3d2e61d6a3a service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:01:57.346669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158045f7-0270-41bf-8923-e3d2e61d6a3a service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:01:57.346812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158045f7-0270-41bf-8923-e3d2e61d6a3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:01:57.356534 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158045f7-0270-41bf-8923-e3d2e61d6a3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:01:57.359285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158045f7-0270-41bf-8923-e3d2e61d6a3a service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:01:57.370373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 699/1409] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:01:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:07.240682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=43 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:02:07.365408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=85 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:02:20.049627 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe0b99-42e7-4589-bc74-fab48ba7c433 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:20.050160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe0b99-42e7-4589-bc74-fab48ba7c433 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:20.050160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe0b99-42e7-4589-bc74-fab48ba7c433 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:20.051482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 711/1410] 10.222.0.45 () {54 vars in 918 bytes} [Tue Jul 5 16:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:20.056517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8985169-00f2-409e-8a06-df515bfeddfc None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:20.056939 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8985169-00f2-409e-8a06-df515bfeddfc None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:20.057098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8985169-00f2-409e-8a06-df515bfeddfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:20.075006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e8985169-00f2-409e-8a06-df515bfeddfc None None] MFA Rules not processed for user `75b7f541993b4a7c915c9e6d440e2674`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:20.076833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e8985169-00f2-409e-8a06-df515bfeddfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:20.085452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 700/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Tue Jul 5 16:02:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:20.099380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f4350-821f-46f0-a6d0-b22041250bf9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:20.099671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f4350-821f-46f0-a6d0-b22041250bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:20.099953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f4350-821f-46f0-a6d0-b22041250bf9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:20.101340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 712/1412] 10.222.0.45 () {54 vars in 918 bytes} [Tue Jul 5 16:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:20.105708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e85f78-9311-4c4d-a382-45c852422eaa None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:20.105998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e85f78-9311-4c4d-a382-45c852422eaa None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:20.106306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e85f78-9311-4c4d-a382-45c852422eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:20.124302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b9e85f78-9311-4c4d-a382-45c852422eaa None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:20.125515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b9e85f78-9311-4c4d-a382-45c852422eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:20.134474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 701/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Tue Jul 5 16:02:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:22.635947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbad4e-7912-4496-a8f8-8fe4e648daef None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:22.636509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbad4e-7912-4496-a8f8-8fe4e648daef None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:22.636509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbad4e-7912-4496-a8f8-8fe4e648daef None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:22.638475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 16:02:22.643742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cafeb8-60a2-413d-96cc-684931d316d9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:22.644175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cafeb8-60a2-413d-96cc-684931d316d9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:22.644307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cafeb8-60a2-413d-96cc-684931d316d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:22.662214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-29cafeb8-60a2-413d-96cc-684931d316d9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:22.663395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-29cafeb8-60a2-413d-96cc-684931d316d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:22.671116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:22.713143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbad4e-7912-4496-a8f8-8fe4e648daef None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:22.727713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbad4e-7912-4496-a8f8-8fe4e648daef None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:22.733349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df9d41-baf3-4b14-8300-1ea73af51dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:22.733970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df9d41-baf3-4b14-8300-1ea73af51dd3 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:22.734253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df9d41-baf3-4b14-8300-1ea73af51dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:22.734528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df9d41-baf3-4b14-8300-1ea73af51dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:22.742400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df9d41-baf3-4b14-8300-1ea73af51dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:22.743994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df9d41-baf3-4b14-8300-1ea73af51dd3 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:22.753604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 714/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:23.646183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2ea06-f153-4079-a429-9ec0dfefe811 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:23.646785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2ea06-f153-4079-a429-9ec0dfefe811 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:23.646785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2ea06-f153-4079-a429-9ec0dfefe811 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:23.648042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:23.652797 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb3366-99d8-420b-b4fc-7c08b2f0181f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:23.653261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb3366-99d8-420b-b4fc-7c08b2f0181f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:23.653400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb3366-99d8-420b-b4fc-7c08b2f0181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:23.671054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-fdbb3366-99d8-420b-b4fc-7c08b2f0181f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:23.672250 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fdbb3366-99d8-420b-b4fc-7c08b2f0181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:23.680101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:23.830002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2ea06-f153-4079-a429-9ec0dfefe811 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:23.844604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2ea06-f153-4079-a429-9ec0dfefe811 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:23.850956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd8c70-c222-4a05-b3d4-49bdf41c53e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:23.850956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd8c70-c222-4a05-b3d4-49bdf41c53e3 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:23.850956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd8c70-c222-4a05-b3d4-49bdf41c53e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:23.851246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd8c70-c222-4a05-b3d4-49bdf41c53e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:23.860000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd8c70-c222-4a05-b3d4-49bdf41c53e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:23.861843 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd8c70-c222-4a05-b3d4-49bdf41c53e3 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:23.871324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 704/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:02:24.746928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8d890-657c-46d6-bbf7-ec0b28fd71ca None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:24.747458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8d890-657c-46d6-bbf7-ec0b28fd71ca None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:24.747555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8d890-657c-46d6-bbf7-ec0b28fd71ca None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:24.748915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:24.753730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4f911-17c9-49ae-82f1-08f66e90da50 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:24.754152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4f911-17c9-49ae-82f1-08f66e90da50 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:24.754261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4f911-17c9-49ae-82f1-08f66e90da50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:24.772180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-0bc4f911-17c9-49ae-82f1-08f66e90da50 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:24.773385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0bc4f911-17c9-49ae-82f1-08f66e90da50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:24.781692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:24.928403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8d890-657c-46d6-bbf7-ec0b28fd71ca None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:24.943090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8d890-657c-46d6-bbf7-ec0b28fd71ca None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:24.949318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69ce66-5a8f-4deb-83c7-46fb0f5196ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:24.949318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ce66-5a8f-4deb-83c7-46fb0f5196ff service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:24.949318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ce66-5a8f-4deb-83c7-46fb0f5196ff service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:24.949604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ce66-5a8f-4deb-83c7-46fb0f5196ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:24.957521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69ce66-5a8f-4deb-83c7-46fb0f5196ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:24.959119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69ce66-5a8f-4deb-83c7-46fb0f5196ff service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:24.968376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 717/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:25.826028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee641f11-19a8-4dd3-a477-fdb1a24c5368 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:25.826608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee641f11-19a8-4dd3-a477-fdb1a24c5368 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:25.826608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee641f11-19a8-4dd3-a477-fdb1a24c5368 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:25.827932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:25.832876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f597feb-2cce-44f0-8173-452d3abf1a22 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:25.833265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f597feb-2cce-44f0-8173-452d3abf1a22 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:25.833471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f597feb-2cce-44f0-8173-452d3abf1a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:25.852408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1f597feb-2cce-44f0-8173-452d3abf1a22 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:25.853631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1f597feb-2cce-44f0-8173-452d3abf1a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:25.861693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:26.010730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee641f11-19a8-4dd3-a477-fdb1a24c5368 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:26.025167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee641f11-19a8-4dd3-a477-fdb1a24c5368 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:26.030451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8056e-1bf5-4b58-b6a7-5bafdc39962c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:26.031076 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8056e-1bf5-4b58-b6a7-5bafdc39962c service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:26.031360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8056e-1bf5-4b58-b6a7-5bafdc39962c service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:26.031638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8056e-1bf5-4b58-b6a7-5bafdc39962c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:26.039588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8056e-1bf5-4b58-b6a7-5bafdc39962c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:26.041192 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8056e-1bf5-4b58-b6a7-5bafdc39962c service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:26.050443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 707/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:26.903040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04854d52-4bef-4ee2-ab94-b028c3547672 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:26.903605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04854d52-4bef-4ee2-ab94-b028c3547672 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:26.903605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04854d52-4bef-4ee2-ab94-b028c3547672 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:26.905047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:26.909913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d783e-391c-4719-a0fb-ec38c1518e1c None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:26.910349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d783e-391c-4719-a0fb-ec38c1518e1c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:26.910489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d783e-391c-4719-a0fb-ec38c1518e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:26.927814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-0c3d783e-391c-4719-a0fb-ec38c1518e1c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:26.929022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0c3d783e-391c-4719-a0fb-ec38c1518e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:26.937078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:27.091588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04854d52-4bef-4ee2-ab94-b028c3547672 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:27.105618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04854d52-4bef-4ee2-ab94-b028c3547672 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:27.112353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c782d5-9976-4683-a505-e01f6ba34b3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:27.113021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c782d5-9976-4683-a505-e01f6ba34b3e service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:27.113307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c782d5-9976-4683-a505-e01f6ba34b3e service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:27.113582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c782d5-9976-4683-a505-e01f6ba34b3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:27.121673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c782d5-9976-4683-a505-e01f6ba34b3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:27.123878 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c782d5-9976-4683-a505-e01f6ba34b3e service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:27.133089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 720/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:28.039484 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ea164-6daf-43bb-beea-0c6a5ba40182 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:28.040012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ea164-6daf-43bb-beea-0c6a5ba40182 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:28.040012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ea164-6daf-43bb-beea-0c6a5ba40182 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:28.041327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:28.046064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1a4b0-97e9-42f1-8225-2624b6d059c0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:28.046482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1a4b0-97e9-42f1-8225-2624b6d059c0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:28.046622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1a4b0-97e9-42f1-8225-2624b6d059c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:28.064281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-7aa1a4b0-97e9-42f1-8225-2624b6d059c0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:28.065511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7aa1a4b0-97e9-42f1-8225-2624b6d059c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:28.073113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:28.223940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ea164-6daf-43bb-beea-0c6a5ba40182 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:28.238180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ea164-6daf-43bb-beea-0c6a5ba40182 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:28.243488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190a5e1-86e2-4e00-9e10-a15aa0e8ef59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:28.244114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190a5e1-86e2-4e00-9e10-a15aa0e8ef59 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:28.244398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190a5e1-86e2-4e00-9e10-a15aa0e8ef59 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:28.244676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190a5e1-86e2-4e00-9e10-a15aa0e8ef59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:28.253325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190a5e1-86e2-4e00-9e10-a15aa0e8ef59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:28.254894 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190a5e1-86e2-4e00-9e10-a15aa0e8ef59 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:28.264972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 710/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:29.159054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ad515-7682-481f-a3c3-8114e72b1100 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:29.159596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ad515-7682-481f-a3c3-8114e72b1100 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:29.159596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ad515-7682-481f-a3c3-8114e72b1100 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:29.161045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 16:02:29.165932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c02e3-3c04-458e-8623-0ec78e1e74cf None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:29.166353 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c02e3-3c04-458e-8623-0ec78e1e74cf None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:29.166513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c02e3-3c04-458e-8623-0ec78e1e74cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:29.184599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-368c02e3-3c04-458e-8623-0ec78e1e74cf None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:29.185788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-368c02e3-3c04-458e-8623-0ec78e1e74cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:29.193373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:29.341904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ad515-7682-481f-a3c3-8114e72b1100 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:29.356195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ad515-7682-481f-a3c3-8114e72b1100 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:29.362089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54638bb-9fe1-4018-a3c6-4444d3bce33c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:29.362727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54638bb-9fe1-4018-a3c6-4444d3bce33c service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:29.363020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54638bb-9fe1-4018-a3c6-4444d3bce33c service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:29.363298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54638bb-9fe1-4018-a3c6-4444d3bce33c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:29.371524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54638bb-9fe1-4018-a3c6-4444d3bce33c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:29.373160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54638bb-9fe1-4018-a3c6-4444d3bce33c service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:29.382559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 723/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:30.280503 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2316a8-f1a0-49cb-96be-62fbefc4b851 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:30.281086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2316a8-f1a0-49cb-96be-62fbefc4b851 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:30.281086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2316a8-f1a0-49cb-96be-62fbefc4b851 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:30.282440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:30.287332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f83dc3-d5e0-4d1a-bda2-06bc4bfa6b63 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:30.287767 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f83dc3-d5e0-4d1a-bda2-06bc4bfa6b63 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:30.287875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f83dc3-d5e0-4d1a-bda2-06bc4bfa6b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:30.306229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-30f83dc3-d5e0-4d1a-bda2-06bc4bfa6b63 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:30.307419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-30f83dc3-d5e0-4d1a-bda2-06bc4bfa6b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:30.315738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:30.463895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2316a8-f1a0-49cb-96be-62fbefc4b851 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:30.478495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2316a8-f1a0-49cb-96be-62fbefc4b851 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:30.484614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a9d41-500b-4ac3-a438-6ff2d3689635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:30.485269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a9d41-500b-4ac3-a438-6ff2d3689635 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:30.485555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a9d41-500b-4ac3-a438-6ff2d3689635 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:30.485834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a9d41-500b-4ac3-a438-6ff2d3689635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:30.493793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a9d41-500b-4ac3-a438-6ff2d3689635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:30.495364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a9d41-500b-4ac3-a438-6ff2d3689635 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:30.504668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 713/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:31.360898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20028f-7ab2-44e7-b23f-376a5845c450 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:31.372027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20028f-7ab2-44e7-b23f-376a5845c450 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:31.372027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20028f-7ab2-44e7-b23f-376a5845c450 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:31.372027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:31.372297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8abd62-67e6-4e06-8206-3c4e4cbc626a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:31.372297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8abd62-67e6-4e06-8206-3c4e4cbc626a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:31.372297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8abd62-67e6-4e06-8206-3c4e4cbc626a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:31.385866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-0f8abd62-67e6-4e06-8206-3c4e4cbc626a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:31.387052 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0f8abd62-67e6-4e06-8206-3c4e4cbc626a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:31.394783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:31.548902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20028f-7ab2-44e7-b23f-376a5845c450 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:31.563441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20028f-7ab2-44e7-b23f-376a5845c450 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:31.568790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b6c9c-16e0-455d-b802-e07d7426d386 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:31.569463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b6c9c-16e0-455d-b802-e07d7426d386 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:31.569744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b6c9c-16e0-455d-b802-e07d7426d386 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:31.570019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b6c9c-16e0-455d-b802-e07d7426d386 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:31.578569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b6c9c-16e0-455d-b802-e07d7426d386 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:31.580151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b6c9c-16e0-455d-b802-e07d7426d386 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:31.589318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 726/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:33.526025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f7035-6d23-4ab0-993a-41ac6aad0a4d None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:33.526620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f7035-6d23-4ab0-993a-41ac6aad0a4d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:33.526620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f7035-6d23-4ab0-993a-41ac6aad0a4d None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:33.527990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:33.533186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a44082-9d38-4ecb-adbb-4407b768140d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:33.533589 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a44082-9d38-4ecb-adbb-4407b768140d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:33.533740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a44082-9d38-4ecb-adbb-4407b768140d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:33.551054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-34a44082-9d38-4ecb-adbb-4407b768140d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:33.552173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-34a44082-9d38-4ecb-adbb-4407b768140d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:33.560544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:33.598283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f7035-6d23-4ab0-993a-41ac6aad0a4d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:33.609077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f7035-6d23-4ab0-993a-41ac6aad0a4d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:33.612529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bfa4f-e1ac-482b-b3a5-7934296bcca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:33.613180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bfa4f-e1ac-482b-b3a5-7934296bcca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:33.613467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bfa4f-e1ac-482b-b3a5-7934296bcca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:33.613742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bfa4f-e1ac-482b-b3a5-7934296bcca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:33.623006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bfa4f-e1ac-482b-b3a5-7934296bcca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:33.624414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bfa4f-e1ac-482b-b3a5-7934296bcca6 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:33.633666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 716/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:35.807567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76ef33-5cd1-4e05-a6d4-c69395ac3ce9 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:35.808131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76ef33-5cd1-4e05-a6d4-c69395ac3ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:35.808131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76ef33-5cd1-4e05-a6d4-c69395ac3ce9 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:35.809567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:35.814814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca994649-7cab-4f83-ad20-8f89ee4cfc23 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:35.815231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca994649-7cab-4f83-ad20-8f89ee4cfc23 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:35.815372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca994649-7cab-4f83-ad20-8f89ee4cfc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:35.833462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-ca994649-7cab-4f83-ad20-8f89ee4cfc23 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:35.834637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ca994649-7cab-4f83-ad20-8f89ee4cfc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:35.842819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:35.880703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76ef33-5cd1-4e05-a6d4-c69395ac3ce9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:35.892219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76ef33-5cd1-4e05-a6d4-c69395ac3ce9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:35.895774 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b028b38-0439-46bb-b009-c4d0050531e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:35.896601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b028b38-0439-46bb-b009-c4d0050531e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:35.896723 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b028b38-0439-46bb-b009-c4d0050531e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:35.896990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b028b38-0439-46bb-b009-c4d0050531e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:35.906632 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b028b38-0439-46bb-b009-c4d0050531e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:35.908067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b028b38-0439-46bb-b009-c4d0050531e7 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:35.917392 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 729/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:38.317789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52475c4f-f957-4e54-a2ec-5336971f28b1 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:38.318235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52475c4f-f957-4e54-a2ec-5336971f28b1 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:38.318356 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52475c4f-f957-4e54-a2ec-5336971f28b1 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:38.319724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:38.324832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e86ac-eb18-4ba5-96a0-d3269027f861 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:38.325749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e86ac-eb18-4ba5-96a0-d3269027f861 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:38.325922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e86ac-eb18-4ba5-96a0-d3269027f861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:38.343823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-df8e86ac-eb18-4ba5-96a0-d3269027f861 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:38.344983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df8e86ac-eb18-4ba5-96a0-d3269027f861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:38.352719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:38.391016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52475c4f-f957-4e54-a2ec-5336971f28b1 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:38.401913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52475c4f-f957-4e54-a2ec-5336971f28b1 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:38.406051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9d0de-3198-4b93-a53b-4da57809e7a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:38.406681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9d0de-3198-4b93-a53b-4da57809e7a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:38.406969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9d0de-3198-4b93-a53b-4da57809e7a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:38.407249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9d0de-3198-4b93-a53b-4da57809e7a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:38.416124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9d0de-3198-4b93-a53b-4da57809e7a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:38.417594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9d0de-3198-4b93-a53b-4da57809e7a0 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:38.427234 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 719/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:40.561046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd46c6-7b25-4140-b6f3-ba8d93ceb425 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:40.561557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd46c6-7b25-4140-b6f3-ba8d93ceb425 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:40.561557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd46c6-7b25-4140-b6f3-ba8d93ceb425 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:40.562900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:40.568064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd5326-d156-4eca-b81d-8930ebc10a47 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:40.568480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd5326-d156-4eca-b81d-8930ebc10a47 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:40.568619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd5326-d156-4eca-b81d-8930ebc10a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:40.585937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1efd5326-d156-4eca-b81d-8930ebc10a47 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:40.587110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1efd5326-d156-4eca-b81d-8930ebc10a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:40.594671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:40.633092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd46c6-7b25-4140-b6f3-ba8d93ceb425 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:40.643592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd46c6-7b25-4140-b6f3-ba8d93ceb425 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:40.647113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a00d9c-7486-4f93-b9d8-5da334d36ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:40.647748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a00d9c-7486-4f93-b9d8-5da334d36ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:40.648034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a00d9c-7486-4f93-b9d8-5da334d36ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:40.648311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a00d9c-7486-4f93-b9d8-5da334d36ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:40.657163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a00d9c-7486-4f93-b9d8-5da334d36ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:40.658608 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a00d9c-7486-4f93-b9d8-5da334d36ba1 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:40.667987 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 732/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:42.722063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7e8c2-5927-4a0f-9548-d5d00841d588 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:42.722590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7e8c2-5927-4a0f-9548-d5d00841d588 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:42.722590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7e8c2-5927-4a0f-9548-d5d00841d588 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:42.724013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:42.729254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c79a3-6a62-41df-ac47-ca2de6947eb1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:42.729661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c79a3-6a62-41df-ac47-ca2de6947eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:42.729802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c79a3-6a62-41df-ac47-ca2de6947eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:42.748300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0a8c79a3-6a62-41df-ac47-ca2de6947eb1 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:42.749505 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0a8c79a3-6a62-41df-ac47-ca2de6947eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:42.757150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:42.794471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7e8c2-5927-4a0f-9548-d5d00841d588 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:42.796162 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-08a7e8c2-5927-4a0f-9548-d5d00841d588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:42.808884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7e8c2-5927-4a0f-9548-d5d00841d588 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:42.813094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a301c-632b-4aa4-82ce-0257ce41b432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:42.813817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a301c-632b-4aa4-82ce-0257ce41b432 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:42.814113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a301c-632b-4aa4-82ce-0257ce41b432 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:42.814389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a301c-632b-4aa4-82ce-0257ce41b432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:42.823396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a301c-632b-4aa4-82ce-0257ce41b432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:42.824850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a301c-632b-4aa4-82ce-0257ce41b432 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:42.834312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 722/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:44.952091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb198b-a3fd-4314-85c6-982a41594106 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:44.952653 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb198b-a3fd-4314-85c6-982a41594106 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:44.952653 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb198b-a3fd-4314-85c6-982a41594106 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:44.954039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:44.959147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772f11-a3af-48af-985e-7a1447e0bac9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:44.959555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772f11-a3af-48af-985e-7a1447e0bac9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:44.959695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772f11-a3af-48af-985e-7a1447e0bac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:44.976893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-fb772f11-a3af-48af-985e-7a1447e0bac9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:44.978078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fb772f11-a3af-48af-985e-7a1447e0bac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:44.986470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:45.023675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb198b-a3fd-4314-85c6-982a41594106 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:45.034633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb198b-a3fd-4314-85c6-982a41594106 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:45.038200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadf105-d4db-4901-8a19-022498c1786e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:45.038830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadf105-d4db-4901-8a19-022498c1786e service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:45.039115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadf105-d4db-4901-8a19-022498c1786e service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:45.039393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadf105-d4db-4901-8a19-022498c1786e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:45.048805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadf105-d4db-4901-8a19-022498c1786e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:45.050231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadf105-d4db-4901-8a19-022498c1786e service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:45.060414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 735/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:47.158983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1760-0c83-4394-a0ac-d0f01c39d2ee None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:47.159559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1760-0c83-4394-a0ac-d0f01c39d2ee None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:47.159559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1760-0c83-4394-a0ac-d0f01c39d2ee None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:47.160910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:47.166143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807b8ec-6026-402d-9deb-f434a9fea043 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:47.166558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807b8ec-6026-402d-9deb-f434a9fea043 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:47.166680 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807b8ec-6026-402d-9deb-f434a9fea043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:47.184434 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e807b8ec-6026-402d-9deb-f434a9fea043 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:47.185650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e807b8ec-6026-402d-9deb-f434a9fea043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:47.193197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:47.233988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd1760-0c83-4394-a0ac-d0f01c39d2ee None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:47.244732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd1760-0c83-4394-a0ac-d0f01c39d2ee None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:47.248286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15143161-da06-43b6-a81d-b9d1c8d3305a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:47.248957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15143161-da06-43b6-a81d-b9d1c8d3305a service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:47.249243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15143161-da06-43b6-a81d-b9d1c8d3305a service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:47.249522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15143161-da06-43b6-a81d-b9d1c8d3305a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:47.258652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15143161-da06-43b6-a81d-b9d1c8d3305a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:47.260044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15143161-da06-43b6-a81d-b9d1c8d3305a service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:47.270646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 725/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:49.411543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e41cf-3947-40d2-9918-1fc4e097fb08 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:49.412072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e41cf-3947-40d2-9918-1fc4e097fb08 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:49.412200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e41cf-3947-40d2-9918-1fc4e097fb08 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:49.413622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:49.419010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76817afd-f21e-4448-ba21-7e5af1e813d5 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:49.419414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76817afd-f21e-4448-ba21-7e5af1e813d5 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:49.419572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76817afd-f21e-4448-ba21-7e5af1e813d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:49.425850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=79 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:02:49.438046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-76817afd-f21e-4448-ba21-7e5af1e813d5 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:49.439346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-76817afd-f21e-4448-ba21-7e5af1e813d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:49.447149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:02:49.485793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e41cf-3947-40d2-9918-1fc4e097fb08 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:49.491256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=73 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:02:49.497523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e41cf-3947-40d2-9918-1fc4e097fb08 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:49.502297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395b1cc-2435-4200-9c58-f2e296f9edd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:49.502297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395b1cc-2435-4200-9c58-f2e296f9edd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:49.502297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395b1cc-2435-4200-9c58-f2e296f9edd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:49.502297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395b1cc-2435-4200-9c58-f2e296f9edd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:49.511295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395b1cc-2435-4200-9c58-f2e296f9edd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:49.512785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395b1cc-2435-4200-9c58-f2e296f9edd1 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:49.522702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 738/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:51.587131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427ebf-5dd5-48bd-8cec-53ed22cdd27e None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:51.587643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427ebf-5dd5-48bd-8cec-53ed22cdd27e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:51.587643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427ebf-5dd5-48bd-8cec-53ed22cdd27e None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:51.589010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:51.594309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d426add-3e0d-44da-9211-fb29fcd9997c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:51.594716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d426add-3e0d-44da-9211-fb29fcd9997c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:51.594857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d426add-3e0d-44da-9211-fb29fcd9997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:51.612383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0d426add-3e0d-44da-9211-fb29fcd9997c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:51.613807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0d426add-3e0d-44da-9211-fb29fcd9997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:51.622276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:51.662550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b427ebf-5dd5-48bd-8cec-53ed22cdd27e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:51.673086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b427ebf-5dd5-48bd-8cec-53ed22cdd27e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:51.676496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0aa4e7-2695-4657-a9da-d4b25d1f3f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:51.677178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0aa4e7-2695-4657-a9da-d4b25d1f3f3d service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:51.677457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0aa4e7-2695-4657-a9da-d4b25d1f3f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:51.677727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0aa4e7-2695-4657-a9da-d4b25d1f3f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:51.686845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0aa4e7-2695-4657-a9da-d4b25d1f3f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:51.688340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0aa4e7-2695-4657-a9da-d4b25d1f3f3d service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:51.697493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 728/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:02:53.789212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da73ae-e700-4ef1-8713-54a5bac336ef None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:53.789779 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da73ae-e700-4ef1-8713-54a5bac336ef None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:53.789779 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da73ae-e700-4ef1-8713-54a5bac336ef None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:53.791147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:02:53.797111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d767721-62d3-43fc-8fb8-eb84ce485b92 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:53.797507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d767721-62d3-43fc-8fb8-eb84ce485b92 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:53.797658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d767721-62d3-43fc-8fb8-eb84ce485b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:53.814910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3d767721-62d3-43fc-8fb8-eb84ce485b92 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:02:53.816089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3d767721-62d3-43fc-8fb8-eb84ce485b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:02:53.823717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:02:53.860977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da73ae-e700-4ef1-8713-54a5bac336ef None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:02:53.871634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da73ae-e700-4ef1-8713-54a5bac336ef None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:02:53.875161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ba493-e998-4b31-8d05-0d4e6b0ca58c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:02:53.875791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ba493-e998-4b31-8d05-0d4e6b0ca58c service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:02:53.876074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ba493-e998-4b31-8d05-0d4e6b0ca58c service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:02:53.876354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ba493-e998-4b31-8d05-0d4e6b0ca58c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:02:53.885146 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ba493-e998-4b31-8d05-0d4e6b0ca58c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:02:53.886571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ba493-e998-4b31-8d05-0d4e6b0ca58c service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:02:53.895990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 741/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:03:03.809992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=11 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:03:03.891908 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=14 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:03:13.615518 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef80f83-78dc-46d6-8a88-b93288c0ec7f None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:13.616041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef80f83-78dc-46d6-8a88-b93288c0ec7f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:13.616041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef80f83-78dc-46d6-8a88-b93288c0ec7f None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:13.617367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 730/1471] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:03:13.622638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd7890-f1ea-4c19-b8aa-a0447bbd238b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:13.623072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd7890-f1ea-4c19-b8aa-a0447bbd238b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:13.623174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd7890-f1ea-4c19-b8aa-a0447bbd238b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:13.640875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-65fd7890-f1ea-4c19-b8aa-a0447bbd238b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:03:13.642053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-65fd7890-f1ea-4c19-b8aa-a0447bbd238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:03:13.649670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 742/1472] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:03:13.733860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01899d90-a36f-4213-b909-98e393e0e17c None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:13.734219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01899d90-a36f-4213-b909-98e393e0e17c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:13.734504 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01899d90-a36f-4213-b909-98e393e0e17c None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:13.735868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 731/1473] 10.222.0.45 () {54 vars in 962 bytes} [Tue Jul 5 16:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:03:13.740574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf85e25-a5c6-4352-81e4-4947a14b1208 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:13.740890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf85e25-a5c6-4352-81e4-4947a14b1208 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:13.741169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf85e25-a5c6-4352-81e4-4947a14b1208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:13.759224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bcf85e25-a5c6-4352-81e4-4947a14b1208 None None] MFA Rules not processed for user `a5b712bd7d8d4930a3bfc99581be8cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:03:13.760403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bcf85e25-a5c6-4352-81e4-4947a14b1208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:03:13.766719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 743/1474] 10.222.0.45 () {58 vars in 1059 bytes} [Tue Jul 5 16:03:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:03:13.773628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01899d90-a36f-4213-b909-98e393e0e17c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:03:13.783941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01899d90-a36f-4213-b909-98e393e0e17c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:03:13.787384 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd89d9-a446-4319-b815-403727967b14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b712bd7d8d4930a3bfc99581be8cb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:03:13.787997 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd89d9-a446-4319-b815-403727967b14 service glance] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:13.788279 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd89d9-a446-4319-b815-403727967b14 service glance] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:13.788554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd89d9-a446-4319-b815-403727967b14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:13.797810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd89d9-a446-4319-b815-403727967b14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:03:13.799221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd89d9-a446-4319-b815-403727967b14 service glance] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:03:13.809433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 732/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:03:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:03:15.907374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e702c4-e32f-42f8-b564-fd826aa05f2c None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:15.907901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e702c4-e32f-42f8-b564-fd826aa05f2c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:15.908021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e702c4-e32f-42f8-b564-fd826aa05f2c None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:15.909369 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 744/1476] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:03:15.914632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f0c6d-6b56-45b1-b120-4ecd832887aa None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:15.915025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f0c6d-6b56-45b1-b120-4ecd832887aa None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:15.915220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f0c6d-6b56-45b1-b120-4ecd832887aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:15.933574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-fe5f0c6d-6b56-45b1-b120-4ecd832887aa None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:03:15.934764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fe5f0c6d-6b56-45b1-b120-4ecd832887aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:03:15.942670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 733/1477] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:03:15.964590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e702c4-e32f-42f8-b564-fd826aa05f2c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:03:15.976618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e702c4-e32f-42f8-b564-fd826aa05f2c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:03:15.981577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119acb77-8252-41aa-a1d8-06dd94d4d57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:03:15.982208 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119acb77-8252-41aa-a1d8-06dd94d4d57e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:15.982491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119acb77-8252-41aa-a1d8-06dd94d4d57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:15.983025 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119acb77-8252-41aa-a1d8-06dd94d4d57e admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:15.983759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119acb77-8252-41aa-a1d8-06dd94d4d57e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:03:15.985561 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-119acb77-8252-41aa-a1d8-06dd94d4d57e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:03:15.985561 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 05 16:03:15.985561 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: Jul 05 16:03:15.986701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119acb77-8252-41aa-a1d8-06dd94d4d57e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:03:16.006174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 745/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Tue Jul 5 16:03:15 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:03:18.080383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf9ba3-e880-48f6-82d2-2ce341cbfba8 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:18.080954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf9ba3-e880-48f6-82d2-2ce341cbfba8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:18.081078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf9ba3-e880-48f6-82d2-2ce341cbfba8 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:18.082408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 734/1479] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:03:18.087674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080343f-64cd-49ca-af97-b31fe66e064b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:18.088093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080343f-64cd-49ca-af97-b31fe66e064b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:18.088260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080343f-64cd-49ca-af97-b31fe66e064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:18.106770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c080343f-64cd-49ca-af97-b31fe66e064b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:03:18.108263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c080343f-64cd-49ca-af97-b31fe66e064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:03:18.116193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 746/1480] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:03:18.158954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf9ba3-e880-48f6-82d2-2ce341cbfba8 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:03:18.172644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf9ba3-e880-48f6-82d2-2ce341cbfba8 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:03:18.178036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca9619-6aee-459b-b940-39d679208c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:03:18.178656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca9619-6aee-459b-b940-39d679208c1c service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:18.178935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca9619-6aee-459b-b940-39d679208c1c service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:18.179208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca9619-6aee-459b-b940-39d679208c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:18.187119 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca9619-6aee-459b-b940-39d679208c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:03:18.188668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca9619-6aee-459b-b940-39d679208c1c service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:03:18.197862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 735/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:03:20.415043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548209d3-e3b6-468d-a74c-d24201bf9a50 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:20.415618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548209d3-e3b6-468d-a74c-d24201bf9a50 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:20.415618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548209d3-e3b6-468d-a74c-d24201bf9a50 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:20.417045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 747/1482] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:03:20.422308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ada7e-e705-4277-9b08-688a6c728cf6 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:20.422705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ada7e-e705-4277-9b08-688a6c728cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:20.422930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ada7e-e705-4277-9b08-688a6c728cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:20.440747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-533ada7e-e705-4277-9b08-688a6c728cf6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:03:20.442000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-533ada7e-e705-4277-9b08-688a6c728cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:03:20.450289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 736/1483] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:03:20.493997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548209d3-e3b6-468d-a74c-d24201bf9a50 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:03:20.504672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548209d3-e3b6-468d-a74c-d24201bf9a50 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:03:20.508128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98616c5a-49ca-4ebe-9043-1eb7161b6d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:03:20.508757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98616c5a-49ca-4ebe-9043-1eb7161b6d8a service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:20.509068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98616c5a-49ca-4ebe-9043-1eb7161b6d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:20.509351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98616c5a-49ca-4ebe-9043-1eb7161b6d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:20.518780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98616c5a-49ca-4ebe-9043-1eb7161b6d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:03:20.520327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98616c5a-49ca-4ebe-9043-1eb7161b6d8a service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:03:20.530263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 748/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:03:22.635154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3663c-f664-4f57-9293-fd70f28a1b10 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:22.635678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3663c-f664-4f57-9293-fd70f28a1b10 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:22.636407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3663c-f664-4f57-9293-fd70f28a1b10 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:22.639412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 737/1485] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:03:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:03:22.644545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe2db2-82d4-463a-b646-b500592eb9aa None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:22.644994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe2db2-82d4-463a-b646-b500592eb9aa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:22.645149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe2db2-82d4-463a-b646-b500592eb9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:22.663543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-49fe2db2-82d4-463a-b646-b500592eb9aa None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:03:22.664745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-49fe2db2-82d4-463a-b646-b500592eb9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:03:22.673083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 749/1486] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:03:22.710202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3663c-f664-4f57-9293-fd70f28a1b10 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:03:22.720655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3663c-f664-4f57-9293-fd70f28a1b10 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:03:22.724111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b702fc0-689a-4186-a72a-b482af8d2268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:03:22.724739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702fc0-689a-4186-a72a-b482af8d2268 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:22.725051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702fc0-689a-4186-a72a-b482af8d2268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:22.725337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b702fc0-689a-4186-a72a-b482af8d2268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:22.734821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b702fc0-689a-4186-a72a-b482af8d2268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:03:22.736266 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b702fc0-689a-4186-a72a-b482af8d2268 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:03:22.746125 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 738/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:03:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:03:24.901433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff25bc-f450-4dff-b2b7-d1a56b6bcd6f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:24.901976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff25bc-f450-4dff-b2b7-d1a56b6bcd6f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:24.901976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff25bc-f450-4dff-b2b7-d1a56b6bcd6f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:24.903419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 750/1488] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:03:24.908706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c5fd1-d5fc-4c42-9384-eec50492004f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:24.909143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c5fd1-d5fc-4c42-9384-eec50492004f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:24.909360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c5fd1-d5fc-4c42-9384-eec50492004f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:24.928274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-048c5fd1-d5fc-4c42-9384-eec50492004f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:03:24.929495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-048c5fd1-d5fc-4c42-9384-eec50492004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:03:24.937727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 739/1489] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:03:24.976645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff25bc-f450-4dff-b2b7-d1a56b6bcd6f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:03:24.988161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff25bc-f450-4dff-b2b7-d1a56b6bcd6f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:03:24.991529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08563270-dc0a-4aec-bc08-732bec1748be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:03:24.992150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08563270-dc0a-4aec-bc08-732bec1748be service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:24.992430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08563270-dc0a-4aec-bc08-732bec1748be service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:24.992704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08563270-dc0a-4aec-bc08-732bec1748be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:25.001961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08563270-dc0a-4aec-bc08-732bec1748be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:03:25.003378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08563270-dc0a-4aec-bc08-732bec1748be service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:03:25.012498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 751/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:03:27.481544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86866106-3467-4762-b7ef-33c0e0aba84b None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:27.482106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86866106-3467-4762-b7ef-33c0e0aba84b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:27.482106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86866106-3467-4762-b7ef-33c0e0aba84b None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:27.483692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 740/1491] 10.222.0.45 () {54 vars in 926 bytes} [Tue Jul 5 16:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:03:27.489172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f320d6-409b-487f-b3f4-e452c10f1540 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:27.489579 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f320d6-409b-487f-b3f4-e452c10f1540 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:27.489726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f320d6-409b-487f-b3f4-e452c10f1540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:27.625478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-08f320d6-409b-487f-b3f4-e452c10f1540 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:03:27.626619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-08f320d6-409b-487f-b3f4-e452c10f1540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:03:27.634723 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 752/1492] 10.222.0.45 () {58 vars in 1023 bytes} [Tue Jul 5 16:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:03:27.673257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86866106-3467-4762-b7ef-33c0e0aba84b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:03:27.684256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86866106-3467-4762-b7ef-33c0e0aba84b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:03:27.687735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3355ed-8253-4433-af45-72ef8f30e0ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:03:27.688357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3355ed-8253-4433-af45-72ef8f30e0ca service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:03:27.688636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3355ed-8253-4433-af45-72ef8f30e0ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:03:27.688945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3355ed-8253-4433-af45-72ef8f30e0ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:03:27.697578 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3355ed-8253-4433-af45-72ef8f30e0ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:03:27.699606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3355ed-8253-4433-af45-72ef8f30e0ca service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:03:27.709548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 741/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:03:37.502963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=33 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:03:37.704714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=33 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:05:09.396988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9008-77e2-4082-a76e-bec1a0211505 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:09.397517 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9008-77e2-4082-a76e-bec1a0211505 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:09.397628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9008-77e2-4082-a76e-bec1a0211505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:09.415807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a6be9008-77e2-4082-a76e-bec1a0211505 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:05:09.417024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a6be9008-77e2-4082-a76e-bec1a0211505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:05:09.425208 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 753/1494] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:05:09.432955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f576b-1a17-4795-a7f2-ea9dc0772545 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:09.433443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f576b-1a17-4795-a7f2-ea9dc0772545 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:09.433546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f576b-1a17-4795-a7f2-ea9dc0772545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:09.451283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f82f576b-1a17-4795-a7f2-ea9dc0772545 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:05:09.452444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f82f576b-1a17-4795-a7f2-ea9dc0772545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:05:09.460126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 742/1495] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:05:09.465229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be9008-77e2-4082-a76e-bec1a0211505 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:05:09.474956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be9008-77e2-4082-a76e-bec1a0211505 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:05:09.479455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5321ac-d829-4a49-87c6-94629f8f4cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:05:09.480102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5321ac-d829-4a49-87c6-94629f8f4cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:09.480380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5321ac-d829-4a49-87c6-94629f8f4cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:09.480651 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5321ac-d829-4a49-87c6-94629f8f4cbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:09.481404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5321ac-d829-4a49-87c6-94629f8f4cbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:05:09.481708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5321ac-d829-4a49-87c6-94629f8f4cbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:05:09.482725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-0f5321ac-d829-4a49-87c6-94629f8f4cbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:05:09.482725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 05 16:05:09.482725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: Jul 05 16:05:09.483877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5321ac-d829-4a49-87c6-94629f8f4cbd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:05:09.495431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 754/1496] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:05:09.503359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f576b-1a17-4795-a7f2-ea9dc0772545 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:05:09.512991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f576b-1a17-4795-a7f2-ea9dc0772545 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:05:09.521868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62b285-3743-4fc8-9f97-b21801afad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:05:09.522518 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62b285-3743-4fc8-9f97-b21801afad2d admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:09.522799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62b285-3743-4fc8-9f97-b21801afad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:09.523859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62b285-3743-4fc8-9f97-b21801afad2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:09.524558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62b285-3743-4fc8-9f97-b21801afad2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:05:09.526539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: WARNING py.warnings [None req-9f62b285-3743-4fc8-9f97-b21801afad2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:05:09.526539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: warnings.warn(msg) Jul 05 16:05:09.526539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: Jul 05 16:05:09.527726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62b285-3743-4fc8-9f97-b21801afad2d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:05:09.617999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 743/1497] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:05:09 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:05:09.625481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5321ac-d829-4a49-87c6-94629f8f4cbd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:05:09.636469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5321ac-d829-4a49-87c6-94629f8f4cbd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:05:09.690634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b42eb9-7013-4679-85bb-e7b9a597d974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:05:09.691287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b42eb9-7013-4679-85bb-e7b9a597d974 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:09.691566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b42eb9-7013-4679-85bb-e7b9a597d974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:09.691837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b42eb9-7013-4679-85bb-e7b9a597d974 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:09.692510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b42eb9-7013-4679-85bb-e7b9a597d974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:05:09.693547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-76b42eb9-7013-4679-85bb-e7b9a597d974 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:05:09.693547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 05 16:05:09.693547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: Jul 05 16:05:09.694732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b42eb9-7013-4679-85bb-e7b9a597d974 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:05:09.793751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 755/1498] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:05:09 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:05:09.797691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62b285-3743-4fc8-9f97-b21801afad2d admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:05:09.808182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62b285-3743-4fc8-9f97-b21801afad2d admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:05:09.812604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c20f64-5ab9-4248-a58e-9b4acdd098f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:05:09.813278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20f64-5ab9-4248-a58e-9b4acdd098f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:09.813566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20f64-5ab9-4248-a58e-9b4acdd098f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:09.813835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20f64-5ab9-4248-a58e-9b4acdd098f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:09.814549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c20f64-5ab9-4248-a58e-9b4acdd098f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:05:09.815549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: WARNING py.warnings [None req-19c20f64-5ab9-4248-a58e-9b4acdd098f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:05:09.815549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: warnings.warn(msg) Jul 05 16:05:09.815549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: Jul 05 16:05:09.816412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c20f64-5ab9-4248-a58e-9b4acdd098f6 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:05:09.830347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 744/1499] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:05:09.834312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b42eb9-7013-4679-85bb-e7b9a597d974 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:05:09.844071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b42eb9-7013-4679-85bb-e7b9a597d974 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:05:09.848538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f0568-8cf9-43d2-9c0a-1660b249a34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:05:09.849221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f0568-8cf9-43d2-9c0a-1660b249a34b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:09.849505 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f0568-8cf9-43d2-9c0a-1660b249a34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:09.849782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f0568-8cf9-43d2-9c0a-1660b249a34b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:09.850512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f0568-8cf9-43d2-9c0a-1660b249a34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:05:09.851863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-2d1f0568-8cf9-43d2-9c0a-1660b249a34b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:05:09.851863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 05 16:05:09.851863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: Jul 05 16:05:09.852741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f0568-8cf9-43d2-9c0a-1660b249a34b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:05:09.866556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 756/1500] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:05:09.871414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c20f64-5ab9-4248-a58e-9b4acdd098f6 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:05:09.887866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c20f64-5ab9-4248-a58e-9b4acdd098f6 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:05:09.892354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288528b-343f-48f6-9252-5d920d2f9220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:05:09.893265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288528b-343f-48f6-9252-5d920d2f9220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:09.893548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288528b-343f-48f6-9252-5d920d2f9220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:09.893823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288528b-343f-48f6-9252-5d920d2f9220 admin admin] PATH_INFO: `/v3/projects/a1cb230a57b347c68c015fab28c870eb/users/286e88d566ef4b308a6c4acfee0854b8/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:09.907966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288528b-343f-48f6-9252-5d920d2f9220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cb230a57b347c68c015fab28c870eb, user_id=286e88d566ef4b308a6c4acfee0854b8, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:05:09.909024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: WARNING py.warnings [None req-9288528b-343f-48f6-9252-5d920d2f9220 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:05:09.909024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: warnings.warn(msg) Jul 05 16:05:09.909024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: Jul 05 16:05:09.910938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288528b-343f-48f6-9252-5d920d2f9220 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:05:10.004934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 745/1501] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:05:09 2022] PUT /identity/v3/projects/a1cb230a57b347c68c015fab28c870eb/users/286e88d566ef4b308a6c4acfee0854b8/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:05:10.017551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd9324-b26b-466c-a296-3c230fcd9b8b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:10.017889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd9324-b26b-466c-a296-3c230fcd9b8b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:10.018170 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd9324-b26b-466c-a296-3c230fcd9b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:10.044390 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e2dd9324-b26b-466c-a296-3c230fcd9b8b None None] MFA Rules not processed for user `286e88d566ef4b308a6c4acfee0854b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:05:10.045631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e2dd9324-b26b-466c-a296-3c230fcd9b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:05:10.106811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 757/1502] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:05:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:05:10.120654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903acb9-be87-4481-9a09-d21f832b26c1 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:10.120978 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903acb9-be87-4481-9a09-d21f832b26c1 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:10.121262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903acb9-be87-4481-9a09-d21f832b26c1 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:10.122672 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 746/1503] 10.222.0.45 () {54 vars in 963 bytes} [Tue Jul 5 16:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:05:10.127318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560c004-2f70-43e4-aef1-331a5f87c907 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:10.127613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560c004-2f70-43e4-aef1-331a5f87c907 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:10.127890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560c004-2f70-43e4-aef1-331a5f87c907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:10.145329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f560c004-2f70-43e4-aef1-331a5f87c907 None None] MFA Rules not processed for user `805a9dd2d0934359b7ca5fe629b96d38`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:05:10.146510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f560c004-2f70-43e4-aef1-331a5f87c907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:05:10.192182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:05:10.199532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 758/1504] 10.222.0.45 () {58 vars in 1060 bytes} [Tue Jul 5 16:05:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:05:10.207951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903acb9-be87-4481-9a09-d21f832b26c1 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:05:10.218656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903acb9-be87-4481-9a09-d21f832b26c1 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:05:10.222155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31734c6-3df1-4d05-a00d-79d66d202781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:05:10.222776 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31734c6-3df1-4d05-a00d-79d66d202781 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:10.223082 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31734c6-3df1-4d05-a00d-79d66d202781 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:10.223351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31734c6-3df1-4d05-a00d-79d66d202781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:10.232172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31734c6-3df1-4d05-a00d-79d66d202781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:05:10.233889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31734c6-3df1-4d05-a00d-79d66d202781 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:05:10.243151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 747/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:05:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:05:10.283540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc8525-a8c7-43fe-80e9-565e36656976 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:10.283836 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc8525-a8c7-43fe-80e9-565e36656976 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:10.284120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc8525-a8c7-43fe-80e9-565e36656976 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:10.285444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 759/1506] 10.222.0.45 () {52 vars in 829 bytes} [Tue Jul 5 16:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:05:10.289422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31734c6-3df1-4d05-a00d-79d66d202781 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:05:10.349894 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31734c6-3df1-4d05-a00d-79d66d202781 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:05:10.354837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:05:10.362255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:10.362255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:10.362255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] PATH_INFO: `/v3/users/286e88d566ef4b308a6c4acfee0854b8` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:10.362255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=286e88d566ef4b308a6c4acfee0854b8)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:05:10.362255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: WARNING py.warnings [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:05:10.362255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: warnings.warn(msg) Jul 05 16:05:10.362255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: Jul 05 16:05:10.362865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:05:10.759840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286e88d566ef4b308a6c4acfee0854b8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:05:10.771715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286e88d566ef4b308a6c4acfee0854b8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:05:10.775198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286e88d566ef4b308a6c4acfee0854b8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:05:10.790902 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286e88d566ef4b308a6c4acfee0854b8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:05:10.791628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286e88d566ef4b308a6c4acfee0854b8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:05:10.795549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:05:10.896392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 748/1507] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:05:10 2022] DELETE /identity/v3/users/286e88d566ef4b308a6c4acfee0854b8 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:05:10.901414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75194-6fc7-4e43-88e1-d0be23c8209e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:10.901905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75194-6fc7-4e43-88e1-d0be23c8209e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:10.902012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75194-6fc7-4e43-88e1-d0be23c8209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:10.919121 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f1e75194-6fc7-4e43-88e1-d0be23c8209e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:05:10.920333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f1e75194-6fc7-4e43-88e1-d0be23c8209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:05:10.993656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 760/1508] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:05:11.003126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:05:11.004545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:05:11.061355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76358a3-9ea8-4c4f-b82d-782354821150 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:05:11.064960 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44822d09-a5c1-45c1-ac67-af059e7239d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:05:11.065654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44822d09-a5c1-45c1-ac67-af059e7239d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:11.065941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44822d09-a5c1-45c1-ac67-af059e7239d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:11.066214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44822d09-a5c1-45c1-ac67-af059e7239d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:11.075758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44822d09-a5c1-45c1-ac67-af059e7239d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:05:11.077215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44822d09-a5c1-45c1-ac67-af059e7239d4 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:05:11.086993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 749/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:05:11.543926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75194-6fc7-4e43-88e1-d0be23c8209e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:05:11.544920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f1e75194-6fc7-4e43-88e1-d0be23c8209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:05:11.557855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75194-6fc7-4e43-88e1-d0be23c8209e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:05:11.562502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:05:11.563364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:05:11.563647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:05:11.563919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] PATH_INFO: `/v3/projects/a1cb230a57b347c68c015fab28c870eb` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:05:11.565253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1cb230a57b347c68c015fab28c870eb)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:05:11.566328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:05:11.566328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 05 16:05:11.566328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: Jul 05 16:05:11.567508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:05:11.690890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1cb230a57b347c68c015fab28c870eb', 'request_id': 'req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:05:11.697317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1cb230a57b347c68c015fab28c870eb', 'request_id': 'req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:05:11.710064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1cb230a57b347c68c015fab28c870eb', 'request_id': 'req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:05:11.805090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] The token cache is being invalidate because project a1cb230a57b347c68c015fab28c870eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:05:11.805419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:05:11.806123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ff8e9e0d-67d7-4d45-a4f0-07845d14c634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:05:11.807656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 761/1510] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:05:11 2022] DELETE /identity/v3/projects/a1cb230a57b347c68c015fab28c870eb => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:05:21.082324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:05:21.801416 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:06:44.935608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897a655-8318-467b-88cc-0241f2ec77b0 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:44.936215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897a655-8318-467b-88cc-0241f2ec77b0 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:44.936215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897a655-8318-467b-88cc-0241f2ec77b0 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:44.937588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:06:44.942474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c3cac-5666-46c3-96c4-a372600701a0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:44.942900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c3cac-5666-46c3-96c4-a372600701a0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:44.943015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c3cac-5666-46c3-96c4-a372600701a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:44.960513 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a94c3cac-5666-46c3-96c4-a372600701a0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:06:44.961756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a94c3cac-5666-46c3-96c4-a372600701a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:06:45.018520 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:06:45.170210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897a655-8318-467b-88cc-0241f2ec77b0 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:06:45.172695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7897a655-8318-467b-88cc-0241f2ec77b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:06:45.245033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897a655-8318-467b-88cc-0241f2ec77b0 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:06:45.250205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7651a-dd9c-40e1-b65c-10641c765121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:06:45.250839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7651a-dd9c-40e1-b65c-10641c765121 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:45.251123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7651a-dd9c-40e1-b65c-10641c765121 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:45.251397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7651a-dd9c-40e1-b65c-10641c765121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:45.259334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7651a-dd9c-40e1-b65c-10641c765121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:06:45.260923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7651a-dd9c-40e1-b65c-10641c765121 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:06:45.270109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 751/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:06:46.097108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ade1b-046f-4a2c-9f9b-a0d4d2a65b92 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:46.097640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ade1b-046f-4a2c-9f9b-a0d4d2a65b92 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:46.097640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ade1b-046f-4a2c-9f9b-a0d4d2a65b92 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:46.098973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:06:46.103815 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21800f82-406a-4941-88d8-e3eb59eb7fd8 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:46.104262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21800f82-406a-4941-88d8-e3eb59eb7fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:46.104392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21800f82-406a-4941-88d8-e3eb59eb7fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:46.122496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-21800f82-406a-4941-88d8-e3eb59eb7fd8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:06:46.123695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-21800f82-406a-4941-88d8-e3eb59eb7fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:06:46.131606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:06:46.282862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ade1b-046f-4a2c-9f9b-a0d4d2a65b92 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:06:46.297141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ade1b-046f-4a2c-9f9b-a0d4d2a65b92 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:06:46.302474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bf6b9-2439-4ab8-81f0-b2bf762f3bcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:06:46.303112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bf6b9-2439-4ab8-81f0-b2bf762f3bcf service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:46.303401 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bf6b9-2439-4ab8-81f0-b2bf762f3bcf service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:46.303681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bf6b9-2439-4ab8-81f0-b2bf762f3bcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:46.312393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bf6b9-2439-4ab8-81f0-b2bf762f3bcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:06:46.314044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bf6b9-2439-4ab8-81f0-b2bf762f3bcf service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:06:46.324280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 764/1516] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:06:46.834523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41f6e1-0d0a-43d4-bf7f-b1d2553c116d None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:46.834837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41f6e1-0d0a-43d4-bf7f-b1d2553c116d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:46.835118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41f6e1-0d0a-43d4-bf7f-b1d2553c116d None None] PATH_INFO: `/v3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:46.836416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 753/1517] 10.222.0.47 () {54 vars in 925 bytes} [Tue Jul 5 16:06:46 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:06:46.842716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551d23a-c6da-4ed8-9679-d81025143fdf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:46.843003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551d23a-c6da-4ed8-9679-d81025143fdf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:46.843280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551d23a-c6da-4ed8-9679-d81025143fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:46.861712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3551d23a-c6da-4ed8-9679-d81025143fdf None None] MFA Rules not processed for user `75b7f541993b4a7c915c9e6d440e2674`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:06:46.862921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3551d23a-c6da-4ed8-9679-d81025143fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:06:46.922376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 765/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Tue Jul 5 16:06:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:06:50.843932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6faa-33ed-4e0d-9a26-600901cfb555 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:50.844432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6faa-33ed-4e0d-9a26-600901cfb555 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:50.844432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6faa-33ed-4e0d-9a26-600901cfb555 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:50.845818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 754/1519] 10.222.0.45 () {54 vars in 960 bytes} [Tue Jul 5 16:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 16:06:50.850812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face5f62-7e72-4daa-9fd8-2fbfcc2384c9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:50.851273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face5f62-7e72-4daa-9fd8-2fbfcc2384c9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:50.851421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face5f62-7e72-4daa-9fd8-2fbfcc2384c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:50.870086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-face5f62-7e72-4daa-9fd8-2fbfcc2384c9 None None] MFA Rules not processed for user `75b7f541993b4a7c915c9e6d440e2674`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:06:50.871266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-face5f62-7e72-4daa-9fd8-2fbfcc2384c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:06:50.879566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 766/1520] 10.222.0.45 () {58 vars in 1057 bytes} [Tue Jul 5 16:06:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:06:50.886732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb6faa-33ed-4e0d-9a26-600901cfb555 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:06:50.900025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb6faa-33ed-4e0d-9a26-600901cfb555 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:06:50.904841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48d5a6-240f-4389-8147-de5b2e493bbe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:06:50.905455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48d5a6-240f-4389-8147-de5b2e493bbe service placement] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:50.905736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48d5a6-240f-4389-8147-de5b2e493bbe service placement] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:50.906012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48d5a6-240f-4389-8147-de5b2e493bbe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:50.913983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48d5a6-240f-4389-8147-de5b2e493bbe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:06:50.916149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48d5a6-240f-4389-8147-de5b2e493bbe service placement] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:06:50.925562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 755/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:06:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:06:57.564530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a48333-279d-4233-a443-164ae2003681 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:57.565102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a48333-279d-4233-a443-164ae2003681 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:57.565102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a48333-279d-4233-a443-164ae2003681 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:57.566431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:06:57.571202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7660f5a-f8b1-4090-98e5-615676a7e90f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:57.571616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7660f5a-f8b1-4090-98e5-615676a7e90f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:57.571807 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7660f5a-f8b1-4090-98e5-615676a7e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:57.590057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b7660f5a-f8b1-4090-98e5-615676a7e90f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:06:57.591251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b7660f5a-f8b1-4090-98e5-615676a7e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:06:57.598966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:06:57.751528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a48333-279d-4233-a443-164ae2003681 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:06:57.765688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a48333-279d-4233-a443-164ae2003681 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:06:57.771063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d1923-1c34-4cae-adc2-4d1ec2155fe9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:06:57.771697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d1923-1c34-4cae-adc2-4d1ec2155fe9 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:57.771984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d1923-1c34-4cae-adc2-4d1ec2155fe9 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:57.772262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d1923-1c34-4cae-adc2-4d1ec2155fe9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:57.781143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d1923-1c34-4cae-adc2-4d1ec2155fe9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:06:57.782735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d1923-1c34-4cae-adc2-4d1ec2155fe9 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:06:57.792756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 768/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:06:58.621698 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acc0e6-ebd6-4e30-bfff-96c747d75323 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:58.622247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acc0e6-ebd6-4e30-bfff-96c747d75323 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:58.622247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acc0e6-ebd6-4e30-bfff-96c747d75323 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:58.623636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 16:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:06:58.628677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf77c9-e2e2-4ccc-a81f-ea7f81d1a41f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:58.629147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf77c9-e2e2-4ccc-a81f-ea7f81d1a41f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:58.629275 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf77c9-e2e2-4ccc-a81f-ea7f81d1a41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:58.647501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-72cf77c9-e2e2-4ccc-a81f-ea7f81d1a41f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:06:58.648689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-72cf77c9-e2e2-4ccc-a81f-ea7f81d1a41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:06:58.656259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 16:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:06:58.808033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acc0e6-ebd6-4e30-bfff-96c747d75323 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:06:58.822817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acc0e6-ebd6-4e30-bfff-96c747d75323 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:06:58.828128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad211ba-859e-4e52-a3cd-57cfb2b3814d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:06:58.828780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad211ba-859e-4e52-a3cd-57cfb2b3814d service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:06:58.829141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad211ba-859e-4e52-a3cd-57cfb2b3814d service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:06:58.829417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad211ba-859e-4e52-a3cd-57cfb2b3814d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:06:58.837594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad211ba-859e-4e52-a3cd-57cfb2b3814d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:06:58.839161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad211ba-859e-4e52-a3cd-57cfb2b3814d service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:06:58.848525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 758/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:07:02.590355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc24062-44f3-4255-9d75-d480160f4b09 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:02.590875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc24062-44f3-4255-9d75-d480160f4b09 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:02.590875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc24062-44f3-4255-9d75-d480160f4b09 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:02.592179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:07:02.597431 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdbffa-0330-4dd1-81d2-4d768307b764 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:02.597857 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdbffa-0330-4dd1-81d2-4d768307b764 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:02.598030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdbffa-0330-4dd1-81d2-4d768307b764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:02.615677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b0cdbffa-0330-4dd1-81d2-4d768307b764 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:07:02.616895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b0cdbffa-0330-4dd1-81d2-4d768307b764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:07:02.624746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:07:02.696293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc24062-44f3-4255-9d75-d480160f4b09 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:07:02.698041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1cc24062-44f3-4255-9d75-d480160f4b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:07:02.755618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc24062-44f3-4255-9d75-d480160f4b09 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:07:02.759641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a53dd1-85a5-4940-9421-6de4dc575ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:07:02.760279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53dd1-85a5-4940-9421-6de4dc575ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:02.760559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53dd1-85a5-4940-9421-6de4dc575ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:02.760860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53dd1-85a5-4940-9421-6de4dc575ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:02.771034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a53dd1-85a5-4940-9421-6de4dc575ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:07:02.772476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a53dd1-85a5-4940-9421-6de4dc575ab5 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:07:02.778075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:07:02.784282 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 771/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:07:04.845824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48749d-b9b3-4589-a9e9-28efd05abb1c None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:04.846367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48749d-b9b3-4589-a9e9-28efd05abb1c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:04.846367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48749d-b9b3-4589-a9e9-28efd05abb1c None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:04.847729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:07:04.853071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a4e01-3ca8-4d5f-8b86-00ec02343799 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:04.853528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a4e01-3ca8-4d5f-8b86-00ec02343799 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:04.853646 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a4e01-3ca8-4d5f-8b86-00ec02343799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:04.872693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cd3a4e01-3ca8-4d5f-8b86-00ec02343799 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:07:04.873919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cd3a4e01-3ca8-4d5f-8b86-00ec02343799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:07:04.883592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:07:04.919522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48749d-b9b3-4589-a9e9-28efd05abb1c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:07:04.930767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48749d-b9b3-4589-a9e9-28efd05abb1c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:07:04.934257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903ad39-49fe-45ef-b28f-b8a33e25e994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:07:04.934890 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903ad39-49fe-45ef-b28f-b8a33e25e994 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:04.935174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903ad39-49fe-45ef-b28f-b8a33e25e994 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:04.935450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903ad39-49fe-45ef-b28f-b8a33e25e994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:04.944922 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903ad39-49fe-45ef-b28f-b8a33e25e994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:07:04.946848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903ad39-49fe-45ef-b28f-b8a33e25e994 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:07:04.956420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 761/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:07:08.515424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637efdd5-7e77-4175-9a9f-68e86ec7ec4a None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:08.516053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637efdd5-7e77-4175-9a9f-68e86ec7ec4a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:08.516053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637efdd5-7e77-4175-9a9f-68e86ec7ec4a None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:08.517750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 16:07:08.523213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8c9d3-f002-474b-9c3f-318834be708b None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:08.523641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8c9d3-f002-474b-9c3f-318834be708b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:08.523773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8c9d3-f002-474b-9c3f-318834be708b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:08.555800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3ef8c9d3-f002-474b-9c3f-318834be708b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:07:08.555800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3ef8c9d3-f002-474b-9c3f-318834be708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:07:08.555800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:07:08.855345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637efdd5-7e77-4175-9a9f-68e86ec7ec4a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:07:08.857032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-637efdd5-7e77-4175-9a9f-68e86ec7ec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:07:08.913036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637efdd5-7e77-4175-9a9f-68e86ec7ec4a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:07:08.917116 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0589b5-f04b-4322-8b67-19471c3df819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:07:08.917748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0589b5-f04b-4322-8b67-19471c3df819 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:08.918033 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0589b5-f04b-4322-8b67-19471c3df819 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:08.918308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0589b5-f04b-4322-8b67-19471c3df819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:08.927112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0589b5-f04b-4322-8b67-19471c3df819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:07:08.928532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0589b5-f04b-4322-8b67-19471c3df819 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:07:08.937725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 774/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:07:10.974822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3d723-3ae5-4dc5-b3fb-d73332a03cf9 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:10.975390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3d723-3ae5-4dc5-b3fb-d73332a03cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:10.975390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3d723-3ae5-4dc5-b3fb-d73332a03cf9 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:10.976732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 16:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:07:10.982041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7112ae2-cec3-4618-a59f-1a38c9b2bcb2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:10.982455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7112ae2-cec3-4618-a59f-1a38c9b2bcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:10.982611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7112ae2-cec3-4618-a59f-1a38c9b2bcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:11.001753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f7112ae2-cec3-4618-a59f-1a38c9b2bcb2 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:07:11.002929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f7112ae2-cec3-4618-a59f-1a38c9b2bcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:07:11.011012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 16:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:07:11.311939 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3d723-3ae5-4dc5-b3fb-d73332a03cf9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:07:11.313653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-26d3d723-3ae5-4dc5-b3fb-d73332a03cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:07:11.321834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=65 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:07:11.327492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3d723-3ae5-4dc5-b3fb-d73332a03cf9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:07:11.331001 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eb87c-1e5a-4e5d-9b33-6d79c2bd3d54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:07:11.331633 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eb87c-1e5a-4e5d-9b33-6d79c2bd3d54 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:07:11.331916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eb87c-1e5a-4e5d-9b33-6d79c2bd3d54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:07:11.332193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eb87c-1e5a-4e5d-9b33-6d79c2bd3d54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:07:11.341478 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29eb87c-1e5a-4e5d-9b33-6d79c2bd3d54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:07:11.348354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29eb87c-1e5a-4e5d-9b33-6d79c2bd3d54 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:07:11.352851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 764/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:07:20.997244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=32 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:07:21.366493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:38.474362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b7dde-47bc-4440-8edf-426dbd85e728 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:38.475003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b7dde-47bc-4440-8edf-426dbd85e728 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:38.475003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b7dde-47bc-4440-8edf-426dbd85e728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:38.497480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e51b7dde-47bc-4440-8edf-426dbd85e728 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:38.498713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e51b7dde-47bc-4440-8edf-426dbd85e728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:38.511064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 776/1540] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:38.519892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e9abc-ad42-43e4-872e-9f7bd047afc5 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:38.520481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e9abc-ad42-43e4-872e-9f7bd047afc5 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:38.520481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e9abc-ad42-43e4-872e-9f7bd047afc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:38.546941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-aa0e9abc-ad42-43e4-872e-9f7bd047afc5 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:38.548159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-aa0e9abc-ad42-43e4-872e-9f7bd047afc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:38.560878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 765/1541] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:38.562420 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b7dde-47bc-4440-8edf-426dbd85e728 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:38.581152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b7dde-47bc-4440-8edf-426dbd85e728 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:38.590120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4873c42-59ce-48b7-b42e-b234cac28be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:38.590821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4873c42-59ce-48b7-b42e-b234cac28be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:38.591106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4873c42-59ce-48b7-b42e-b234cac28be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:38.591385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4873c42-59ce-48b7-b42e-b234cac28be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:38.592139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4873c42-59ce-48b7-b42e-b234cac28be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:38.592452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4873c42-59ce-48b7-b42e-b234cac28be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:38.594242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4873c42-59ce-48b7-b42e-b234cac28be1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:38.614453 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 777/1542] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:08:38.622686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e9abc-ad42-43e4-872e-9f7bd047afc5 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:38.640806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e9abc-ad42-43e4-872e-9f7bd047afc5 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:38.645753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585c7d4-ce19-4dbb-9d96-812594d66444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:38.646414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585c7d4-ce19-4dbb-9d96-812594d66444 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:38.646691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585c7d4-ce19-4dbb-9d96-812594d66444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:38.646963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585c7d4-ce19-4dbb-9d96-812594d66444 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:38.647653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585c7d4-ce19-4dbb-9d96-812594d66444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:38.650212 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585c7d4-ce19-4dbb-9d96-812594d66444 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:38.744626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 766/1543] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:38 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:38.749751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4873c42-59ce-48b7-b42e-b234cac28be1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:38.764295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4873c42-59ce-48b7-b42e-b234cac28be1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:38.851019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7efcd1-f49b-4008-be3f-6aa36d7cf020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:38.851814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7efcd1-f49b-4008-be3f-6aa36d7cf020 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:38.852110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7efcd1-f49b-4008-be3f-6aa36d7cf020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:38.852392 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7efcd1-f49b-4008-be3f-6aa36d7cf020 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:38.853148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7efcd1-f49b-4008-be3f-6aa36d7cf020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:38.855539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7efcd1-f49b-4008-be3f-6aa36d7cf020 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:38.887395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19e4d2-9833-444c-bc5d-77cac0bf3530 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:38.887733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19e4d2-9833-444c-bc5d-77cac0bf3530 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:38.887993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19e4d2-9833-444c-bc5d-77cac0bf3530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:38.919255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-5f19e4d2-9833-444c-bc5d-77cac0bf3530 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:38.920489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5f19e4d2-9833-444c-bc5d-77cac0bf3530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:38.938032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 767/1544] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:08:38.939545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55c49b-3833-4c8c-932c-375309650eb3 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:38.939835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55c49b-3833-4c8c-932c-375309650eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:38.940107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55c49b-3833-4c8c-932c-375309650eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:38.970721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3a55c49b-3833-4c8c-932c-375309650eb3 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:38.971909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3a55c49b-3833-4c8c-932c-375309650eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:38.987310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 768/1545] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:38.990641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 778/1546] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:38 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:38.990761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31c229-0feb-44dc-93e6-17a6181b4493 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:38.990761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31c229-0feb-44dc-93e6-17a6181b4493 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:38.990761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31c229-0feb-44dc-93e6-17a6181b4493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:38.990993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a45357-678a-48f7-b4bb-97f148c6d97e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:38.990993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a45357-678a-48f7-b4bb-97f148c6d97e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:38.990993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a45357-678a-48f7-b4bb-97f148c6d97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.022328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-92a45357-678a-48f7-b4bb-97f148c6d97e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.023646 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-92a45357-678a-48f7-b4bb-97f148c6d97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.025748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-0d31c229-0feb-44dc-93e6-17a6181b4493 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.026915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0d31c229-0feb-44dc-93e6-17a6181b4493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.039776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 779/1547] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.040448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a45357-678a-48f7-b4bb-97f148c6d97e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.047049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 769/1548] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.048250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb348827-9290-4daf-87f6-905fcf40b1ba None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.048541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb348827-9290-4daf-87f6-905fcf40b1ba None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.048832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb348827-9290-4daf-87f6-905fcf40b1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.056589 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a45357-678a-48f7-b4bb-97f148c6d97e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.071506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a95ff-7bd9-4daa-aafc-aff5da8addaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.072212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a95ff-7bd9-4daa-aafc-aff5da8addaa admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.072483 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a95ff-7bd9-4daa-aafc-aff5da8addaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.072794 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a95ff-7bd9-4daa-aafc-aff5da8addaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.073573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a95ff-7bd9-4daa-aafc-aff5da8addaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.080745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a95ff-7bd9-4daa-aafc-aff5da8addaa admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.086385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-fb348827-9290-4daf-87f6-905fcf40b1ba None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.087636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fb348827-9290-4daf-87f6-905fcf40b1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.099536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 780/1549] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:39.100299 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 770/1550] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.100691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f33431-d0de-4411-a90a-b851698e85bb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.101126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f33431-d0de-4411-a90a-b851698e85bb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.101246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb348827-9290-4daf-87f6-905fcf40b1ba None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.102354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f33431-d0de-4411-a90a-b851698e85bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.125033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb348827-9290-4daf-87f6-905fcf40b1ba None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.134596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-05f33431-d0de-4411-a90a-b851698e85bb None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.135817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-05f33431-d0de-4411-a90a-b851698e85bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.139236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a336ff-59b8-486a-b050-cacdb7a1ad56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.139953 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a336ff-59b8-486a-b050-cacdb7a1ad56 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.140239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a336ff-59b8-486a-b050-cacdb7a1ad56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.140516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a336ff-59b8-486a-b050-cacdb7a1ad56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.141278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a336ff-59b8-486a-b050-cacdb7a1ad56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:39.141760 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a336ff-59b8-486a-b050-cacdb7a1ad56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.143064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: WARNING py.warnings [None req-15a336ff-59b8-486a-b050-cacdb7a1ad56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:08:39.143064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: warnings.warn(msg) Jul 05 16:08:39.143064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: Jul 05 16:08:39.143952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a336ff-59b8-486a-b050-cacdb7a1ad56 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.158266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 781/1551] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.159573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436411d-01c9-4b33-a7f9-30b694f65c7d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.159862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436411d-01c9-4b33-a7f9-30b694f65c7d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.160141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436411d-01c9-4b33-a7f9-30b694f65c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.169965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 771/1552] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:08:39.171750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a336ff-59b8-486a-b050-cacdb7a1ad56 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.189787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a336ff-59b8-486a-b050-cacdb7a1ad56 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.193719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9436411d-01c9-4b33-a7f9-30b694f65c7d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.194560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5731e-c144-4a22-bfc7-117a53d3cea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.194930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9436411d-01c9-4b33-a7f9-30b694f65c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.195263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5731e-c144-4a22-bfc7-117a53d3cea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.195554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5731e-c144-4a22-bfc7-117a53d3cea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.195839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5731e-c144-4a22-bfc7-117a53d3cea0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.196577 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5731e-c144-4a22-bfc7-117a53d3cea0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:39.196933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5731e-c144-4a22-bfc7-117a53d3cea0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.198739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5731e-c144-4a22-bfc7-117a53d3cea0 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.204452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 782/1553] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.205458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436411d-01c9-4b33-a7f9-30b694f65c7d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.215871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 772/1554] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:08:39.216746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5731e-c144-4a22-bfc7-117a53d3cea0 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.221268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436411d-01c9-4b33-a7f9-30b694f65c7d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.226182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba72e8-b85e-4897-8a7f-c996b99d3c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.227406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba72e8-b85e-4897-8a7f-c996b99d3c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.227689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba72e8-b85e-4897-8a7f-c996b99d3c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.227965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba72e8-b85e-4897-8a7f-c996b99d3c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.228513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5731e-c144-4a22-bfc7-117a53d3cea0 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.228706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba72e8-b85e-4897-8a7f-c996b99d3c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.230565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba72e8-b85e-4897-8a7f-c996b99d3c3c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.233609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d2a92-241c-4923-a680-bef0308be274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.234542 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d2a92-241c-4923-a680-bef0308be274 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.234830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d2a92-241c-4923-a680-bef0308be274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.235110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d2a92-241c-4923-a680-bef0308be274 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.235926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d2a92-241c-4923-a680-bef0308be274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:39.236239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d2a92-241c-4923-a680-bef0308be274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.237851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d2a92-241c-4923-a680-bef0308be274 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.244006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 783/1555] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:39.244970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba72e8-b85e-4897-8a7f-c996b99d3c3c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.252127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 773/1556] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:08:39.253570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60341597-28b5-4ffa-9717-0ea3d2da90fe None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.253871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60341597-28b5-4ffa-9717-0ea3d2da90fe None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.254158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60341597-28b5-4ffa-9717-0ea3d2da90fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.254719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba72e8-b85e-4897-8a7f-c996b99d3c3c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.259223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6df66-6027-4450-8f90-197b38ec171f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.259859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6df66-6027-4450-8f90-197b38ec171f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.260209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6df66-6027-4450-8f90-197b38ec171f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.260404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6df66-6027-4450-8f90-197b38ec171f admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.261125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6df66-6027-4450-8f90-197b38ec171f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.262153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-ebd6df66-6027-4450-8f90-197b38ec171f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:08:39.262153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 05 16:08:39.262153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: Jul 05 16:08:39.263333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6df66-6027-4450-8f90-197b38ec171f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.271985 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-60341597-28b5-4ffa-9717-0ea3d2da90fe None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.273302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-60341597-28b5-4ffa-9717-0ea3d2da90fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.281363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 774/1557] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.282727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345192b-513a-4785-9518-679bfc02275f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.283026 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345192b-513a-4785-9518-679bfc02275f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.283305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345192b-513a-4785-9518-679bfc02275f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.302724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-5345192b-513a-4785-9518-679bfc02275f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.303884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5345192b-513a-4785-9518-679bfc02275f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.312440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 775/1558] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.313213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345192b-513a-4785-9518-679bfc02275f None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.322412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345192b-513a-4785-9518-679bfc02275f None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.326857 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c43ae-d05a-47d6-82ea-2506c59bf040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.327474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c43ae-d05a-47d6-82ea-2506c59bf040 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.327761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c43ae-d05a-47d6-82ea-2506c59bf040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.328041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c43ae-d05a-47d6-82ea-2506c59bf040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.328718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c43ae-d05a-47d6-82ea-2506c59bf040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.330666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c43ae-d05a-47d6-82ea-2506c59bf040 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.366442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 784/1559] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/projects => generated 393 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:39.369028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767851cd-c590-4e30-837e-b9a791e139ed None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.369319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767851cd-c590-4e30-837e-b9a791e139ed None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.369593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767851cd-c590-4e30-837e-b9a791e139ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.386846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-767851cd-c590-4e30-837e-b9a791e139ed None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.388086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-767851cd-c590-4e30-837e-b9a791e139ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.427197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 776/1560] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:39.427197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ee7c9-f025-4f09-8828-d1697e246ed7 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.427197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ee7c9-f025-4f09-8828-d1697e246ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.427197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ee7c9-f025-4f09-8828-d1697e246ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.443874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8c8ee7c9-f025-4f09-8828-d1697e246ed7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.445094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8c8ee7c9-f025-4f09-8828-d1697e246ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.455317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 785/1561] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.456306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767851cd-c590-4e30-837e-b9a791e139ed None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.465932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767851cd-c590-4e30-837e-b9a791e139ed None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.469787 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:39.511489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 777/1562] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.512312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ee7c9-f025-4f09-8828-d1697e246ed7 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.521336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ee7c9-f025-4f09-8828-d1697e246ed7 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.523879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f843cb-08a3-4bf4-a78a-78c09cf01faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.524805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f843cb-08a3-4bf4-a78a-78c09cf01faa admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.525073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f843cb-08a3-4bf4-a78a-78c09cf01faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.525353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f843cb-08a3-4bf4-a78a-78c09cf01faa admin admin] PATH_INFO: `/v3/projects/55e522ac286d4dc584d658f850f49d0a/users/967d6158df3a4ff696a80d27c1f7483a/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.526428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e601338-5151-4124-9ecf-30340670d59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.527071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601338-5151-4124-9ecf-30340670d59b admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.527357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601338-5151-4124-9ecf-30340670d59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.527630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601338-5151-4124-9ecf-30340670d59b admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.528315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e601338-5151-4124-9ecf-30340670d59b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.530244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e601338-5151-4124-9ecf-30340670d59b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.541755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f843cb-08a3-4bf4-a78a-78c09cf01faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e522ac286d4dc584d658f850f49d0a, user_id=967d6158df3a4ff696a80d27c1f7483a, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.542801 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-09f843cb-08a3-4bf4-a78a-78c09cf01faa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:08:39.542801 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 05 16:08:39.542801 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: Jul 05 16:08:39.544783 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f843cb-08a3-4bf4-a78a-78c09cf01faa admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.618560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 778/1563] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:39.619773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee629e6-e9d4-4773-9627-38adb2a2055c None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.620066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee629e6-e9d4-4773-9627-38adb2a2055c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.620361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee629e6-e9d4-4773-9627-38adb2a2055c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.637561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-aee629e6-e9d4-4773-9627-38adb2a2055c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.638835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-aee629e6-e9d4-4773-9627-38adb2a2055c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.642888 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:39.664568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 786/1564] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:39 2022] PUT /identity/v3/projects/55e522ac286d4dc584d658f850f49d0a/users/967d6158df3a4ff696a80d27c1f7483a/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:39.666939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f843cb-08a3-4bf4-a78a-78c09cf01faa admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.698911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 779/1565] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.699727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee629e6-e9d4-4773-9627-38adb2a2055c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.708655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee629e6-e9d4-4773-9627-38adb2a2055c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.735368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f843cb-08a3-4bf4-a78a-78c09cf01faa admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.740144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc2058-3f40-4297-99a5-2c364972ce91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.740828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc2058-3f40-4297-99a5-2c364972ce91 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.741106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc2058-3f40-4297-99a5-2c364972ce91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.741375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc2058-3f40-4297-99a5-2c364972ce91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.742126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc2058-3f40-4297-99a5-2c364972ce91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:39.742438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc2058-3f40-4297-99a5-2c364972ce91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.744047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc2058-3f40-4297-99a5-2c364972ce91 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.755344 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 787/1566] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:08:39.756114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc2058-3f40-4297-99a5-2c364972ce91 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.765421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc2058-3f40-4297-99a5-2c364972ce91 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.769863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fada3-10a2-4790-9d1e-c22108a1ef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.770492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fada3-10a2-4790-9d1e-c22108a1ef39 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.770773 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fada3-10a2-4790-9d1e-c22108a1ef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.771050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fada3-10a2-4790-9d1e-c22108a1ef39 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.771719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fada3-10a2-4790-9d1e-c22108a1ef39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.771973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ceefc-b478-4232-9c25-ee7edffdf6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.772643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ceefc-b478-4232-9c25-ee7edffdf6ee admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.772960 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ceefc-b478-4232-9c25-ee7edffdf6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.773913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ceefc-b478-4232-9c25-ee7edffdf6ee admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.774197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fada3-10a2-4790-9d1e-c22108a1ef39 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.774605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ceefc-b478-4232-9c25-ee7edffdf6ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.775617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: WARNING py.warnings [None req-463ceefc-b478-4232-9c25-ee7edffdf6ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:08:39.775617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: warnings.warn(msg) Jul 05 16:08:39.775617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: Jul 05 16:08:39.776830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ceefc-b478-4232-9c25-ee7edffdf6ee admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.885155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 788/1567] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:39.886427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991752c-6268-499e-86c8-656aa78cb14a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.886720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991752c-6268-499e-86c8-656aa78cb14a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.887007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991752c-6268-499e-86c8-656aa78cb14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.897869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 780/1568] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:39.899869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f21495-989b-4795-8070-9e48e79a6cb7 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.901750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f21495-989b-4795-8070-9e48e79a6cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.901750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f21495-989b-4795-8070-9e48e79a6cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.906944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9991752c-6268-499e-86c8-656aa78cb14a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.908136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9991752c-6268-499e-86c8-656aa78cb14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.916545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 789/1569] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.917323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991752c-6268-499e-86c8-656aa78cb14a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:39.922730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-89f21495-989b-4795-8070-9e48e79a6cb7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.923953 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-89f21495-989b-4795-8070-9e48e79a6cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:39.926323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991752c-6268-499e-86c8-656aa78cb14a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:39.931293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0cd39-40ae-4ee5-b2fe-5b2190ba37cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:39.931926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0cd39-40ae-4ee5-b2fe-5b2190ba37cf admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.932205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0cd39-40ae-4ee5-b2fe-5b2190ba37cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.932344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 781/1570] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:39.932486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0cd39-40ae-4ee5-b2fe-5b2190ba37cf admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.933166 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0cd39-40ae-4ee5-b2fe-5b2190ba37cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:39.933563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4aca63-6178-44fd-846f-c998d52cf841 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:39.933846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4aca63-6178-44fd-846f-c998d52cf841 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:39.934120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4aca63-6178-44fd-846f-c998d52cf841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:39.935073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0cd39-40ae-4ee5-b2fe-5b2190ba37cf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:39.959154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-5a4aca63-6178-44fd-846f-c998d52cf841 None None] MFA Rules not processed for user `967d6158df3a4ff696a80d27c1f7483a`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:39.960325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5a4aca63-6178-44fd-846f-c998d52cf841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:40.022159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 782/1571] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:40.022977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4aca63-6178-44fd-846f-c998d52cf841 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.028459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 790/1572] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:39 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:40.029327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0cd39-40ae-4ee5-b2fe-5b2190ba37cf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.033146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4aca63-6178-44fd-846f-c998d52cf841 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.037614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b01a09-57e8-4918-a8fa-308e64f96e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.038256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b01a09-57e8-4918-a8fa-308e64f96e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.038537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b01a09-57e8-4918-a8fa-308e64f96e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.039374 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b01a09-57e8-4918-a8fa-308e64f96e10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.039544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0cd39-40ae-4ee5-b2fe-5b2190ba37cf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.040097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b01a09-57e8-4918-a8fa-308e64f96e10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:40.040400 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b01a09-57e8-4918-a8fa-308e64f96e10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.042019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b01a09-57e8-4918-a8fa-308e64f96e10 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.044255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f3a35-ba73-4a1b-803c-24426bd48c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.044951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f3a35-ba73-4a1b-803c-24426bd48c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.045215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f3a35-ba73-4a1b-803c-24426bd48c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.045492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f3a35-ba73-4a1b-803c-24426bd48c50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.046178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f3a35-ba73-4a1b-803c-24426bd48c50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.048105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f3a35-ba73-4a1b-803c-24426bd48c50 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.053733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 783/1573] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:08:40.060661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b01a09-57e8-4918-a8fa-308e64f96e10 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.070396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b01a09-57e8-4918-a8fa-308e64f96e10 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.074934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46506ef-33d3-4313-a801-5372573ef493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.075571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46506ef-33d3-4313-a801-5372573ef493 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.075850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46506ef-33d3-4313-a801-5372573ef493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.076120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46506ef-33d3-4313-a801-5372573ef493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.076994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46506ef-33d3-4313-a801-5372573ef493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.078558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46506ef-33d3-4313-a801-5372573ef493 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.080581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:40.089410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 784/1574] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:40.090399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46506ef-33d3-4313-a801-5372573ef493 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.099609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46506ef-33d3-4313-a801-5372573ef493 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.104679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba75e3b-04c6-4435-be63-131d07b5399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.106256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba75e3b-04c6-4435-be63-131d07b5399b admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.106256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba75e3b-04c6-4435-be63-131d07b5399b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.106256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba75e3b-04c6-4435-be63-131d07b5399b admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.106644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba75e3b-04c6-4435-be63-131d07b5399b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.108121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba75e3b-04c6-4435-be63-131d07b5399b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.118775 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 785/1575] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:40.119605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba75e3b-04c6-4435-be63-131d07b5399b admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.129247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba75e3b-04c6-4435-be63-131d07b5399b admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.131594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 791/1576] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:40 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:40.132554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f3a35-ba73-4a1b-803c-24426bd48c50 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.137299 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428658c-1d00-4882-8280-f368b357081a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.137951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428658c-1d00-4882-8280-f368b357081a admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.138236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428658c-1d00-4882-8280-f368b357081a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.138511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428658c-1d00-4882-8280-f368b357081a admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.139360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428658c-1d00-4882-8280-f368b357081a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:40.139678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428658c-1d00-4882-8280-f368b357081a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.141418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428658c-1d00-4882-8280-f368b357081a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.153567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 786/1577] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:08:40.154346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428658c-1d00-4882-8280-f368b357081a admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.162408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:40.164282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428658c-1d00-4882-8280-f368b357081a admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.194728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f3a35-ba73-4a1b-803c-24426bd48c50 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.199379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54cd1d-68df-4fdd-be99-3668ebdb60ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.200029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54cd1d-68df-4fdd-be99-3668ebdb60ed admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.200309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54cd1d-68df-4fdd-be99-3668ebdb60ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.200580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54cd1d-68df-4fdd-be99-3668ebdb60ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.201330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54cd1d-68df-4fdd-be99-3668ebdb60ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:40.201830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54cd1d-68df-4fdd-be99-3668ebdb60ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.203467 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54cd1d-68df-4fdd-be99-3668ebdb60ed admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.215812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 792/1578] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:08:40.217296 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54cd1d-68df-4fdd-be99-3668ebdb60ed admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.222750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189b478-df16-44a8-a983-e207fc2001db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.231272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189b478-df16-44a8-a983-e207fc2001db admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.231272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189b478-df16-44a8-a983-e207fc2001db admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.231272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189b478-df16-44a8-a983-e207fc2001db admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.231272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189b478-df16-44a8-a983-e207fc2001db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.231272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189b478-df16-44a8-a983-e207fc2001db admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.231752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54cd1d-68df-4fdd-be99-3668ebdb60ed admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.232289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76791d-7a9b-42ca-bf03-fb079a26b519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.232999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76791d-7a9b-42ca-bf03-fb079a26b519 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.233284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76791d-7a9b-42ca-bf03-fb079a26b519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.233560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76791d-7a9b-42ca-bf03-fb079a26b519 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.234536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76791d-7a9b-42ca-bf03-fb079a26b519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.236097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76791d-7a9b-42ca-bf03-fb079a26b519 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.246700 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 793/1579] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:40.247526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76791d-7a9b-42ca-bf03-fb079a26b519 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.256963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76791d-7a9b-42ca-bf03-fb079a26b519 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.261506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aecb9-92e3-4b05-b8f7-7561f8a051a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.262122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aecb9-92e3-4b05-b8f7-7561f8a051a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.262402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aecb9-92e3-4b05-b8f7-7561f8a051a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.262675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aecb9-92e3-4b05-b8f7-7561f8a051a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.263343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aecb9-92e3-4b05-b8f7-7561f8a051a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.265217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aecb9-92e3-4b05-b8f7-7561f8a051a6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.314420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 787/1580] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:40 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:40.317376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189b478-df16-44a8-a983-e207fc2001db admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.371837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 794/1581] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:40 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:40.372451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aecb9-92e3-4b05-b8f7-7561f8a051a6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.377778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189b478-df16-44a8-a983-e207fc2001db admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.432730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f4bc2-1b78-447d-916a-fefa61c010f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.433434 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f4bc2-1b78-447d-916a-fefa61c010f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.433719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f4bc2-1b78-447d-916a-fefa61c010f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.433994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f4bc2-1b78-447d-916a-fefa61c010f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.434719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f4bc2-1b78-447d-916a-fefa61c010f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.436709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f4bc2-1b78-447d-916a-fefa61c010f7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.440039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aecb9-92e3-4b05-b8f7-7561f8a051a6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.445886 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed02a3-25cd-418d-bae3-5362b2ec99d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.446569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed02a3-25cd-418d-bae3-5362b2ec99d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.446849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed02a3-25cd-418d-bae3-5362b2ec99d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.447122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed02a3-25cd-418d-bae3-5362b2ec99d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.447852 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed02a3-25cd-418d-bae3-5362b2ec99d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.449328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 788/1582] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:40.449466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed02a3-25cd-418d-bae3-5362b2ec99d4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.450121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f4bc2-1b78-447d-916a-fefa61c010f7 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.455229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:40.460189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f4bc2-1b78-447d-916a-fefa61c010f7 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.462561 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 795/1583] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:40.463283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed02a3-25cd-418d-bae3-5362b2ec99d4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.470941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8c69f-33ed-4f1c-99d5-3adeb0be62b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.470941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8c69f-33ed-4f1c-99d5-3adeb0be62b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.470941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8c69f-33ed-4f1c-99d5-3adeb0be62b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.470941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8c69f-33ed-4f1c-99d5-3adeb0be62b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.471305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8c69f-33ed-4f1c-99d5-3adeb0be62b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.471305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8c69f-33ed-4f1c-99d5-3adeb0be62b7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.472453 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed02a3-25cd-418d-bae3-5362b2ec99d4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.477693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb11874-6875-4645-8dc1-4f240b8fceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.478317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb11874-6875-4645-8dc1-4f240b8fceae admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.478599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb11874-6875-4645-8dc1-4f240b8fceae admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.478874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb11874-6875-4645-8dc1-4f240b8fceae admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.479552 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb11874-6875-4645-8dc1-4f240b8fceae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.481451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb11874-6875-4645-8dc1-4f240b8fceae admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.583740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 789/1584] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:40 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:40.584692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8c69f-33ed-4f1c-99d5-3adeb0be62b7 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.595109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8c69f-33ed-4f1c-99d5-3adeb0be62b7 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.600308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249a512-a1c7-4709-a794-0e2701067f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.600969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249a512-a1c7-4709-a794-0e2701067f9f admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.601265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249a512-a1c7-4709-a794-0e2701067f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.601546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249a512-a1c7-4709-a794-0e2701067f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.602219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249a512-a1c7-4709-a794-0e2701067f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.602585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 796/1585] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:40 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:40.603352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb11874-6875-4645-8dc1-4f240b8fceae admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.604303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249a512-a1c7-4709-a794-0e2701067f9f admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.661898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:40.666813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb11874-6875-4645-8dc1-4f240b8fceae admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.671450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5017616-25f4-470e-b974-f8ef7008d51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.672145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5017616-25f4-470e-b974-f8ef7008d51f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.672435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5017616-25f4-470e-b974-f8ef7008d51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.672709 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5017616-25f4-470e-b974-f8ef7008d51f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.673454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5017616-25f4-470e-b974-f8ef7008d51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.675057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5017616-25f4-470e-b974-f8ef7008d51f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.684668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 797/1586] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:40.685548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5017616-25f4-470e-b974-f8ef7008d51f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.694828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5017616-25f4-470e-b974-f8ef7008d51f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.699800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbdcc4-77f8-483e-8927-da49d37a61b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.700407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbdcc4-77f8-483e-8927-da49d37a61b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.700687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbdcc4-77f8-483e-8927-da49d37a61b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.700990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbdcc4-77f8-483e-8927-da49d37a61b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.701649 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbdcc4-77f8-483e-8927-da49d37a61b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.703487 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbdcc4-77f8-483e-8927-da49d37a61b0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.708924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 790/1587] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:40 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:40.708924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249a512-a1c7-4709-a794-0e2701067f9f admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.769116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249a512-a1c7-4709-a794-0e2701067f9f admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.774247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e506556-7cee-47ba-b7ed-f0c724b7a725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.774894 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e506556-7cee-47ba-b7ed-f0c724b7a725 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.775174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e506556-7cee-47ba-b7ed-f0c724b7a725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.775446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e506556-7cee-47ba-b7ed-f0c724b7a725 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.776121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e506556-7cee-47ba-b7ed-f0c724b7a725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.778054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e506556-7cee-47ba-b7ed-f0c724b7a725 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.783895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 798/1588] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:40 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:40.786659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbdcc4-77f8-483e-8927-da49d37a61b0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.794900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbdcc4-77f8-483e-8927-da49d37a61b0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.799458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168582f-a2b5-4887-b4b4-eb967a1c335f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.800517 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168582f-a2b5-4887-b4b4-eb967a1c335f admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.801016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168582f-a2b5-4887-b4b4-eb967a1c335f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.801599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168582f-a2b5-4887-b4b4-eb967a1c335f admin admin] PATH_INFO: `/v3/projects/3ec37a457530439b8728dd2ad2b6b7bd/users/d7e2fc1ec676438392d7925b43b806f5/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.816076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168582f-a2b5-4887-b4b4-eb967a1c335f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec37a457530439b8728dd2ad2b6b7bd, user_id=d7e2fc1ec676438392d7925b43b806f5, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.818831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168582f-a2b5-4887-b4b4-eb967a1c335f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.869048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 791/1589] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:40 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:40.869841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e506556-7cee-47ba-b7ed-f0c724b7a725 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.878879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e506556-7cee-47ba-b7ed-f0c724b7a725 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.883846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e4842-e144-48e6-927c-6d1510a2b83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.884725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4842-e144-48e6-927c-6d1510a2b83a admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.885043 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4842-e144-48e6-927c-6d1510a2b83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.885317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4842-e144-48e6-927c-6d1510a2b83a admin admin] PATH_INFO: `/v3/projects/bbe6a2de3cba4aa7b365f30f5b2f745c/users/8b6e665914774eabba120e986e6bb282/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.892509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 799/1590] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:40 2022] PUT /identity/v3/projects/3ec37a457530439b8728dd2ad2b6b7bd/users/d7e2fc1ec676438392d7925b43b806f5/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:40.893649 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168582f-a2b5-4887-b4b4-eb967a1c335f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.905046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e4842-e144-48e6-927c-6d1510a2b83a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe6a2de3cba4aa7b365f30f5b2f745c, user_id=8b6e665914774eabba120e986e6bb282, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.907808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e4842-e144-48e6-927c-6d1510a2b83a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.954906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168582f-a2b5-4887-b4b4-eb967a1c335f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:40.959515 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7305cd3-7cf1-45c4-9665-29d0140763d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:40.960196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7305cd3-7cf1-45c4-9665-29d0140763d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:40.960476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7305cd3-7cf1-45c4-9665-29d0140763d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:40.960792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7305cd3-7cf1-45c4-9665-29d0140763d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:40.961535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7305cd3-7cf1-45c4-9665-29d0140763d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:40.963225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7305cd3-7cf1-45c4-9665-29d0140763d6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:40.971876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 792/1591] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:40 2022] PUT /identity/v3/projects/bbe6a2de3cba4aa7b365f30f5b2f745c/users/8b6e665914774eabba120e986e6bb282/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:40.972705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e4842-e144-48e6-927c-6d1510a2b83a admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.973470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 800/1592] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:40.974296 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7305cd3-7cf1-45c4-9665-29d0140763d6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:40.984321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7305cd3-7cf1-45c4-9665-29d0140763d6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.040446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de76f7-4cef-4af8-b74a-e1299a841de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.040684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e4842-e144-48e6-927c-6d1510a2b83a admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.041351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de76f7-4cef-4af8-b74a-e1299a841de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.041630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de76f7-4cef-4af8-b74a-e1299a841de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.041901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de76f7-4cef-4af8-b74a-e1299a841de5 admin admin] PATH_INFO: `/v3/projects/986d1a42da2042aba9eff3da2dc9b12e/users/e65a088d77fd4a55b619c98fa751bb01/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.045118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008d4ca-eb8e-4f43-b0c1-acad74f4b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.045799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008d4ca-eb8e-4f43-b0c1-acad74f4b223 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.046077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008d4ca-eb8e-4f43-b0c1-acad74f4b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.046350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008d4ca-eb8e-4f43-b0c1-acad74f4b223 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.047029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008d4ca-eb8e-4f43-b0c1-acad74f4b223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.048959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008d4ca-eb8e-4f43-b0c1-acad74f4b223 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.056350 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de76f7-4cef-4af8-b74a-e1299a841de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=986d1a42da2042aba9eff3da2dc9b12e, user_id=e65a088d77fd4a55b619c98fa751bb01, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.059048 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de76f7-4cef-4af8-b74a-e1299a841de5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.115263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 793/1593] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:40 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:41.116086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008d4ca-eb8e-4f43-b0c1-acad74f4b223 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.126061 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008d4ca-eb8e-4f43-b0c1-acad74f4b223 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.126393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 801/1594] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:40 2022] PUT /identity/v3/projects/986d1a42da2042aba9eff3da2dc9b12e/users/e65a088d77fd4a55b619c98fa751bb01/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:41.127270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de76f7-4cef-4af8-b74a-e1299a841de5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.145986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:41.166845 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:41.188679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e73a75-4c3d-4dec-a625-6ee1a54228a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.189526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e73a75-4c3d-4dec-a625-6ee1a54228a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.189816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e73a75-4c3d-4dec-a625-6ee1a54228a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.190099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e73a75-4c3d-4dec-a625-6ee1a54228a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.190792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e73a75-4c3d-4dec-a625-6ee1a54228a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.192756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e73a75-4c3d-4dec-a625-6ee1a54228a0 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.199744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de76f7-4cef-4af8-b74a-e1299a841de5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.204447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bd44d-afd2-4b91-ae2b-f375b4bf5e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.211665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd44d-afd2-4b91-ae2b-f375b4bf5e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.211665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd44d-afd2-4b91-ae2b-f375b4bf5e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.211665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bd44d-afd2-4b91-ae2b-f375b4bf5e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.211665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bd44d-afd2-4b91-ae2b-f375b4bf5e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.211665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bd44d-afd2-4b91-ae2b-f375b4bf5e24 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.217741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 802/1595] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:41.218509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bd44d-afd2-4b91-ae2b-f375b4bf5e24 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.227466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bd44d-afd2-4b91-ae2b-f375b4bf5e24 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.232139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461ffdc-7ceb-4bb0-be18-f88787ece35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.232992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461ffdc-7ceb-4bb0-be18-f88787ece35b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.233646 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461ffdc-7ceb-4bb0-be18-f88787ece35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.234249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461ffdc-7ceb-4bb0-be18-f88787ece35b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.234967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461ffdc-7ceb-4bb0-be18-f88787ece35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.236515 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461ffdc-7ceb-4bb0-be18-f88787ece35b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.246673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 803/1596] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:41.247799 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184bb3-c9cd-4395-91ef-f973413ffedb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.248087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184bb3-c9cd-4395-91ef-f973413ffedb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.248364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184bb3-c9cd-4395-91ef-f973413ffedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.266283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3f184bb3-c9cd-4395-91ef-f973413ffedb None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:41.267504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3f184bb3-c9cd-4395-91ef-f973413ffedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:41.273258 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 794/1597] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:41 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:41.274559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459f4f1-7e9f-4e02-ab89-5c067ef3ac2a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.274850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459f4f1-7e9f-4e02-ab89-5c067ef3ac2a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.275162 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459f4f1-7e9f-4e02-ab89-5c067ef3ac2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.280657 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 804/1598] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:41.280657 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f184bb3-c9cd-4395-91ef-f973413ffedb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.289215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f184bb3-c9cd-4395-91ef-f973413ffedb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.295694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69764b-95e2-41a9-8af9-33181e558260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.296336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c459f4f1-7e9f-4e02-ab89-5c067ef3ac2a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:41.296468 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69764b-95e2-41a9-8af9-33181e558260 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.296620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69764b-95e2-41a9-8af9-33181e558260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.296918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69764b-95e2-41a9-8af9-33181e558260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.297593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c459f4f1-7e9f-4e02-ab89-5c067ef3ac2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:41.297703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69764b-95e2-41a9-8af9-33181e558260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.299218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69764b-95e2-41a9-8af9-33181e558260 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.305746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 795/1599] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:41.306510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459f4f1-7e9f-4e02-ab89-5c067ef3ac2a None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.313402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 805/1600] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:41.313402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8f101-5749-44e8-9154-f0b19d4663ab None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.313402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8f101-5749-44e8-9154-f0b19d4663ab None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.313402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8f101-5749-44e8-9154-f0b19d4663ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.318265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459f4f1-7e9f-4e02-ab89-5c067ef3ac2a None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.323544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c354d6e-655c-48e6-b7c5-4a0ab38ad674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.324206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c354d6e-655c-48e6-b7c5-4a0ab38ad674 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.326577 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c354d6e-655c-48e6-b7c5-4a0ab38ad674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.326577 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c354d6e-655c-48e6-b7c5-4a0ab38ad674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.326577 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c354d6e-655c-48e6-b7c5-4a0ab38ad674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.327135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c354d6e-655c-48e6-b7c5-4a0ab38ad674 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.337375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 796/1601] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:41.338155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c354d6e-655c-48e6-b7c5-4a0ab38ad674 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.341868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6fa8f101-5749-44e8-9154-f0b19d4663ab None None] MFA Rules not processed for user `e65a088d77fd4a55b619c98fa751bb01`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:41.343057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6fa8f101-5749-44e8-9154-f0b19d4663ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:41.347569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c354d6e-655c-48e6-b7c5-4a0ab38ad674 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.352604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eaab7-2191-4820-b7b1-2e7761dfeb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.353488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eaab7-2191-4820-b7b1-2e7761dfeb09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.353789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eaab7-2191-4820-b7b1-2e7761dfeb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.354049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eaab7-2191-4820-b7b1-2e7761dfeb09 admin admin] PATH_INFO: `/v3/projects/23c254877bfa424e963e4491565f828f/users/23c522c6cd614eae9738ba4cbca41a3a/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.368924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0eaab7-2191-4820-b7b1-2e7761dfeb09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c254877bfa424e963e4491565f828f, user_id=23c522c6cd614eae9738ba4cbca41a3a, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.371543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0eaab7-2191-4820-b7b1-2e7761dfeb09 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.407889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 806/1602] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:41.408723 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8f101-5749-44e8-9154-f0b19d4663ab None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.419102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8f101-5749-44e8-9154-f0b19d4663ab None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.423802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35424cd4-7664-45a4-a4fc-5f1da327eb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.427853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35424cd4-7664-45a4-a4fc-5f1da327eb49 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.427853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35424cd4-7664-45a4-a4fc-5f1da327eb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.427853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35424cd4-7664-45a4-a4fc-5f1da327eb49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.427853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35424cd4-7664-45a4-a4fc-5f1da327eb49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.427853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35424cd4-7664-45a4-a4fc-5f1da327eb49 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.441547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 807/1603] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:41.441547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35424cd4-7664-45a4-a4fc-5f1da327eb49 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.449059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35424cd4-7664-45a4-a4fc-5f1da327eb49 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.453988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62abc3c-75ba-40fa-8b1e-eb585cddb9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.454636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62abc3c-75ba-40fa-8b1e-eb585cddb9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.454921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62abc3c-75ba-40fa-8b1e-eb585cddb9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.455193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62abc3c-75ba-40fa-8b1e-eb585cddb9b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.455900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62abc3c-75ba-40fa-8b1e-eb585cddb9b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.457572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62abc3c-75ba-40fa-8b1e-eb585cddb9b0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.467381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 808/1604] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:41.468248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62abc3c-75ba-40fa-8b1e-eb585cddb9b0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.478092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 797/1605] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:41 2022] PUT /identity/v3/projects/23c254877bfa424e963e4491565f828f/users/23c522c6cd614eae9738ba4cbca41a3a/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:41.480232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eaab7-2191-4820-b7b1-2e7761dfeb09 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.525482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62abc3c-75ba-40fa-8b1e-eb585cddb9b0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.543231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:41.545630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eaab7-2191-4820-b7b1-2e7761dfeb09 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.550219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581665f-ae4d-416a-802c-a5236681eaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.551104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581665f-ae4d-416a-802c-a5236681eaed admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.551387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581665f-ae4d-416a-802c-a5236681eaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.551665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581665f-ae4d-416a-802c-a5236681eaed admin admin] PATH_INFO: `/v3/projects/42e6ad67dbf542528726c545872ac10c/users/0c8512e7c1704fac9cc4fcfe3bf1f18b/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.566878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581665f-ae4d-416a-802c-a5236681eaed admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e6ad67dbf542528726c545872ac10c, user_id=0c8512e7c1704fac9cc4fcfe3bf1f18b, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.569573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581665f-ae4d-416a-802c-a5236681eaed admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.575793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7794a7f-3c95-47b1-946a-f5f037065756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.576447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7794a7f-3c95-47b1-946a-f5f037065756 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.576736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7794a7f-3c95-47b1-946a-f5f037065756 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.577043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7794a7f-3c95-47b1-946a-f5f037065756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.586137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7794a7f-3c95-47b1-946a-f5f037065756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.587563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7794a7f-3c95-47b1-946a-f5f037065756 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.597164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 809/1606] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:41.597956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7794a7f-3c95-47b1-946a-f5f037065756 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.605954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7794a7f-3c95-47b1-946a-f5f037065756 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.610070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ec2e-59eb-43f4-b7ad-0bb670f95fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.610891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ec2e-59eb-43f4-b7ad-0bb670f95fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.611200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ec2e-59eb-43f4-b7ad-0bb670f95fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.611474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ec2e-59eb-43f4-b7ad-0bb670f95fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.621225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ec2e-59eb-43f4-b7ad-0bb670f95fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.622663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ec2e-59eb-43f4-b7ad-0bb670f95fa0 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.631889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 810/1607] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:41.632644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ec2e-59eb-43f4-b7ad-0bb670f95fa0 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.641193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ec2e-59eb-43f4-b7ad-0bb670f95fa0 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.645768 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283bc7c-f9f6-459e-b222-fe41da3f930b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.646396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283bc7c-f9f6-459e-b222-fe41da3f930b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.646682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283bc7c-f9f6-459e-b222-fe41da3f930b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.646954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283bc7c-f9f6-459e-b222-fe41da3f930b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.647654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283bc7c-f9f6-459e-b222-fe41da3f930b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.649241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283bc7c-f9f6-459e-b222-fe41da3f930b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.659069 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 811/1608] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:41.660301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc4e78-2242-410e-b829-e72ac012fc63 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.660589 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc4e78-2242-410e-b829-e72ac012fc63 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.660889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc4e78-2242-410e-b829-e72ac012fc63 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.662425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 812/1609] 10.222.0.45 () {54 vars in 961 bytes} [Tue Jul 5 16:08:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:08:41.662898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc4e78-2242-410e-b829-e72ac012fc63 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.668739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 798/1610] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:41 2022] PUT /identity/v3/projects/42e6ad67dbf542528726c545872ac10c/users/0c8512e7c1704fac9cc4fcfe3bf1f18b/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:41.670487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581665f-ae4d-416a-802c-a5236681eaed admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.721020 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:41.736810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581665f-ae4d-416a-802c-a5236681eaed admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.737756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc4e78-2242-410e-b829-e72ac012fc63 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.742047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac338b4-5661-44b5-b98d-de4a7c97235b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.742360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f5e6b-a59f-44f8-bc18-178bdc746091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.742696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac338b4-5661-44b5-b98d-de4a7c97235b admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.743116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac338b4-5661-44b5-b98d-de4a7c97235b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.743239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f5e6b-a59f-44f8-bc18-178bdc746091 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.743412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac338b4-5661-44b5-b98d-de4a7c97235b admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.743510 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f5e6b-a59f-44f8-bc18-178bdc746091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.743770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f5e6b-a59f-44f8-bc18-178bdc746091 admin admin] PATH_INFO: `/v3/projects/b6acb89542314578839f701251add904/users/cb0474d4e6dd4d2b9918fd0ebe803dcd/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.744287 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac338b4-5661-44b5-b98d-de4a7c97235b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.746040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac338b4-5661-44b5-b98d-de4a7c97235b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.758574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 799/1611] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:41.759416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac338b4-5661-44b5-b98d-de4a7c97235b admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.762933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f5e6b-a59f-44f8-bc18-178bdc746091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6acb89542314578839f701251add904, user_id=cb0474d4e6dd4d2b9918fd0ebe803dcd, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.765693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f5e6b-a59f-44f8-bc18-178bdc746091 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.770025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac338b4-5661-44b5-b98d-de4a7c97235b admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.775455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ff868-f64b-40e4-afc7-100dcfe338dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.776113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ff868-f64b-40e4-afc7-100dcfe338dd admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.776396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ff868-f64b-40e4-afc7-100dcfe338dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.776671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ff868-f64b-40e4-afc7-100dcfe338dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.777433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ff868-f64b-40e4-afc7-100dcfe338dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.779144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ff868-f64b-40e4-afc7-100dcfe338dd admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.789901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 800/1612] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:41.791539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ff868-f64b-40e4-afc7-100dcfe338dd admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:41.799968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ff868-f64b-40e4-afc7-100dcfe338dd admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:41.804549 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1f1c-39dd-4889-a675-8c37e7ed0ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:41.805439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1f1c-39dd-4889-a675-8c37e7ed0ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.805720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1f1c-39dd-4889-a675-8c37e7ed0ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.805992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be1f1c-39dd-4889-a675-8c37e7ed0ffe admin admin] PATH_INFO: `/v3/projects/aba0375144644b01b15fb971eaf8b184/users/b2497e2db101402e80478eac693d8953/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.821412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be1f1c-39dd-4889-a675-8c37e7ed0ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba0375144644b01b15fb971eaf8b184, user_id=b2497e2db101402e80478eac693d8953, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:41.824056 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be1f1c-39dd-4889-a675-8c37e7ed0ffe admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:41.844942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 813/1613] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:41 2022] PUT /identity/v3/projects/b6acb89542314578839f701251add904/users/cb0474d4e6dd4d2b9918fd0ebe803dcd/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:41.849039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba223fa-5f48-460e-8b5f-3ff7d8222e1e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.849039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba223fa-5f48-460e-8b5f-3ff7d8222e1e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.849039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba223fa-5f48-460e-8b5f-3ff7d8222e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.865534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3ba223fa-5f48-460e-8b5f-3ff7d8222e1e None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:41.867327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3ba223fa-5f48-460e-8b5f-3ff7d8222e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:41.930452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 814/1614] 10.222.0.45 () {58 vars in 1058 bytes} [Tue Jul 5 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:41.931710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c73f-d16d-432d-aa0b-688865c7ebc5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:41.932000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c73f-d16d-432d-aa0b-688865c7ebc5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:41.932278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c73f-d16d-432d-aa0b-688865c7ebc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:41.948319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ed00c73f-d16d-432d-aa0b-688865c7ebc5 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:41.949514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ed00c73f-d16d-432d-aa0b-688865c7ebc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:41.968417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 801/1615] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:41 2022] PUT /identity/v3/projects/aba0375144644b01b15fb971eaf8b184/users/b2497e2db101402e80478eac693d8953/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:41.969539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1f1c-39dd-4889-a675-8c37e7ed0ffe admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:42.028469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:42.033881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 815/1616] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:42.035473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00c73f-d16d-432d-aa0b-688865c7ebc5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:42.046757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00c73f-d16d-432d-aa0b-688865c7ebc5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:42.051068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be1f1c-39dd-4889-a675-8c37e7ed0ffe admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:42.051969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22f8f5-71a7-45b6-809d-6a20aed2e8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:42.053013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22f8f5-71a7-45b6-809d-6a20aed2e8b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.058448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22f8f5-71a7-45b6-809d-6a20aed2e8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.058448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22f8f5-71a7-45b6-809d-6a20aed2e8b8 admin admin] PATH_INFO: `/v3/projects/23c254877bfa424e963e4491565f828f/users/23c522c6cd614eae9738ba4cbca41a3a/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.058448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22f8f5-71a7-45b6-809d-6a20aed2e8b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c254877bfa424e963e4491565f828f, user_id=23c522c6cd614eae9738ba4cbca41a3a, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:42.058733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2a5ed-68f8-47d3-9c46-7b63adb688ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:42.058733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a5ed-68f8-47d3-9c46-7b63adb688ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.058733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a5ed-68f8-47d3-9c46-7b63adb688ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.058733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a5ed-68f8-47d3-9c46-7b63adb688ae admin admin] PATH_INFO: `/v3/projects/ba872aaf0fd84b1eac87e18659d222d5/users/d89990bfd774417ab0eb52b91ec560d8/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.059076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22f8f5-71a7-45b6-809d-6a20aed2e8b8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:42.073220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 816/1617] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:42 2022] PUT /identity/v3/projects/23c254877bfa424e963e4491565f828f/users/23c522c6cd614eae9738ba4cbca41a3a/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:42.073220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535dba25-1e00-456c-a3c2-cb4c100772c0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.073220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535dba25-1e00-456c-a3c2-cb4c100772c0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.073220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535dba25-1e00-456c-a3c2-cb4c100772c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.080621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2a5ed-68f8-47d3-9c46-7b63adb688ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba872aaf0fd84b1eac87e18659d222d5, user_id=d89990bfd774417ab0eb52b91ec560d8, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:42.083874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2a5ed-68f8-47d3-9c46-7b63adb688ae admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:42.104339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-535dba25-1e00-456c-a3c2-cb4c100772c0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:42.104339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-535dba25-1e00-456c-a3c2-cb4c100772c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:42.157175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 802/1618] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:41 2022] PUT /identity/v3/projects/ba872aaf0fd84b1eac87e18659d222d5/users/d89990bfd774417ab0eb52b91ec560d8/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:42.161726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2a5ed-68f8-47d3-9c46-7b63adb688ae admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:42.163973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 817/1619] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:42.165482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fcc09-1acc-4410-9c90-6cf2cc0f0101 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.165886 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fcc09-1acc-4410-9c90-6cf2cc0f0101 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.166256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833fcc09-1acc-4410-9c90-6cf2cc0f0101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.199586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-833fcc09-1acc-4410-9c90-6cf2cc0f0101 None None] MFA Rules not processed for user `b2497e2db101402e80478eac693d8953`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:42.200868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-833fcc09-1acc-4410-9c90-6cf2cc0f0101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:42.242734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2a5ed-68f8-47d3-9c46-7b63adb688ae admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:42.248220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07dcfc-e3f1-4f66-b255-eae37ea469bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:42.249035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07dcfc-e3f1-4f66-b255-eae37ea469bf service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.249390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07dcfc-e3f1-4f66-b255-eae37ea469bf service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.249676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07dcfc-e3f1-4f66-b255-eae37ea469bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.268581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 818/1620] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:42.274868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fcc09-1acc-4410-9c90-6cf2cc0f0101 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:42.276693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-833fcc09-1acc-4410-9c90-6cf2cc0f0101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:42.310477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:42.323268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07dcfc-e3f1-4f66-b255-eae37ea469bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:42.325103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07dcfc-e3f1-4f66-b255-eae37ea469bf service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:42.334872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 803/1621] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:42.336716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9194-e800-45b6-a831-d22494b26fa9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.337110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9194-e800-45b6-a831-d22494b26fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.337375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9194-e800-45b6-a831-d22494b26fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.351039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833fcc09-1acc-4410-9c90-6cf2cc0f0101 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:42.354977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab346350-7ea5-4703-aeb5-cbf00d4a4c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:42.355799 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab346350-7ea5-4703-aeb5-cbf00d4a4c2a service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.356209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab346350-7ea5-4703-aeb5-cbf00d4a4c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.356530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab346350-7ea5-4703-aeb5-cbf00d4a4c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.364683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:42.364828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e4da9194-e800-45b6-a831-d22494b26fa9 None None] MFA Rules not processed for user `23c522c6cd614eae9738ba4cbca41a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:42.366181 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e4da9194-e800-45b6-a831-d22494b26fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:42.441087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab346350-7ea5-4703-aeb5-cbf00d4a4c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:42.449900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab346350-7ea5-4703-aeb5-cbf00d4a4c2a service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:42.462233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 819/1622] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:42.470252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0322f4b-e771-450b-938e-e18a6f69d049 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.470252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0322f4b-e771-450b-938e-e18a6f69d049 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.470252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0322f4b-e771-450b-938e-e18a6f69d049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.494200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b0322f4b-e771-450b-938e-e18a6f69d049 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:42.495518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b0322f4b-e771-450b-938e-e18a6f69d049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:42.500331 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 804/1623] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:42.501489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da9194-e800-45b6-a831-d22494b26fa9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:42.511257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 820/1624] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:42.512809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6a928-4cca-4cdd-be2c-9973c00f69b0 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.513206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6a928-4cca-4cdd-be2c-9973c00f69b0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.513840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6a928-4cca-4cdd-be2c-9973c00f69b0 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.515005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da9194-e800-45b6-a831-d22494b26fa9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:42.516300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 821/1625] 10.222.0.45 () {54 vars in 961 bytes} [Tue Jul 5 16:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:08:42.517228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6a928-4cca-4cdd-be2c-9973c00f69b0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:42.519154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcecd2-9a07-4cd2-8d1b-d7ae3bde289b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:42.519810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcecd2-9a07-4cd2-8d1b-d7ae3bde289b service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.520274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcecd2-9a07-4cd2-8d1b-d7ae3bde289b service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.521107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcecd2-9a07-4cd2-8d1b-d7ae3bde289b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.532817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcecd2-9a07-4cd2-8d1b-d7ae3bde289b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:42.534525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcecd2-9a07-4cd2-8d1b-d7ae3bde289b service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:42.545383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 805/1626] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:42.546758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8c2ff-6071-4be3-828c-b45cb5c79f55 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.547037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8c2ff-6071-4be3-828c-b45cb5c79f55 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.547362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8c2ff-6071-4be3-828c-b45cb5c79f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.571818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f0c8c2ff-6071-4be3-828c-b45cb5c79f55 None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:42.573561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f0c8c2ff-6071-4be3-828c-b45cb5c79f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:42.585078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 806/1627] 10.222.0.45 () {58 vars in 1058 bytes} [Tue Jul 5 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:42.586245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8c2ff-6071-4be3-828c-b45cb5c79f55 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:42.597169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8c2ff-6071-4be3-828c-b45cb5c79f55 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:42.597844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6a928-4cca-4cdd-be2c-9973c00f69b0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:42.602102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfe823-25d9-4348-9eb2-44ee5bd77a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:42.602918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfe823-25d9-4348-9eb2-44ee5bd77a48 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.603473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10edc01-35e8-4c7c-8506-c0a04d499bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:42.603684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfe823-25d9-4348-9eb2-44ee5bd77a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.604160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfe823-25d9-4348-9eb2-44ee5bd77a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.604691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10edc01-35e8-4c7c-8506-c0a04d499bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.608329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10edc01-35e8-4c7c-8506-c0a04d499bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.609223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10edc01-35e8-4c7c-8506-c0a04d499bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.623714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10edc01-35e8-4c7c-8506-c0a04d499bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:42.624504 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfe823-25d9-4348-9eb2-44ee5bd77a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:42.625396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10edc01-35e8-4c7c-8506-c0a04d499bb7 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:42.626990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfe823-25d9-4348-9eb2-44ee5bd77a48 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:42.639794 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 822/1628] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:42.640849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10edc01-35e8-4c7c-8506-c0a04d499bb7 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:42.648183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 807/1629] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:42.655472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10edc01-35e8-4c7c-8506-c0a04d499bb7 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:42.661696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec33779-2392-4387-b945-5391485e1b8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:42.662435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec33779-2392-4387-b945-5391485e1b8e service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.662784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec33779-2392-4387-b945-5391485e1b8e service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.663227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec33779-2392-4387-b945-5391485e1b8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.674625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec33779-2392-4387-b945-5391485e1b8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:42.676312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec33779-2392-4387-b945-5391485e1b8e service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:42.688016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 823/1630] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:42.879175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c8cfd-8dc0-4c9f-8c5e-b1eca823b821 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.880198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c8cfd-8dc0-4c9f-8c5e-b1eca823b821 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.880488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c8cfd-8dc0-4c9f-8c5e-b1eca823b821 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.882177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 808/1631] 10.222.0.45 () {54 vars in 912 bytes} [Tue Jul 5 16:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:08:42.891749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52b663-d437-4557-9629-55f171826253 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.892039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52b663-d437-4557-9629-55f171826253 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.892315 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52b663-d437-4557-9629-55f171826253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.916411 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ff52b663-d437-4557-9629-55f171826253 None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:42.917726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ff52b663-d437-4557-9629-55f171826253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:42.928091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 824/1632] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:42.941514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c8cfd-8dc0-4c9f-8c5e-b1eca823b821 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:42.953040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c8cfd-8dc0-4c9f-8c5e-b1eca823b821 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:42.957324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92aa7f-5459-4877-959c-d5b9ee1e0e97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:42.958224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92aa7f-5459-4877-959c-d5b9ee1e0e97 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.958711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92aa7f-5459-4877-959c-d5b9ee1e0e97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.959228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92aa7f-5459-4877-959c-d5b9ee1e0e97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.970256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de8b3d-ca67-4b4b-b87d-1c22902272d2 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.970536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de8b3d-ca67-4b4b-b87d-1c22902272d2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.970866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de8b3d-ca67-4b4b-b87d-1c22902272d2 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:42.972355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 825/1633] 10.222.0.45 () {54 vars in 912 bytes} [Tue Jul 5 16:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:08:42.973176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92aa7f-5459-4877-959c-d5b9ee1e0e97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:42.974983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92aa7f-5459-4877-959c-d5b9ee1e0e97 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:42.983035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0951c-f1a1-4caf-8b7e-443bbe19fd94 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:42.983363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0951c-f1a1-4caf-8b7e-443bbe19fd94 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:42.983631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0951c-f1a1-4caf-8b7e-443bbe19fd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:43.005026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c5c0951c-f1a1-4caf-8b7e-443bbe19fd94 None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:43.006533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c5c0951c-f1a1-4caf-8b7e-443bbe19fd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:43.016468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 809/1634] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:43.017212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 826/1635] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:08:43.089004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92aa7f-5459-4877-959c-d5b9ee1e0e97 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:43.123204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92aa7f-5459-4877-959c-d5b9ee1e0e97 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:43.137234 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e1481-6e05-465c-843f-1ee59df11b1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:43.138233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1481-6e05-465c-843f-1ee59df11b1b service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:43.144492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1481-6e05-465c-843f-1ee59df11b1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:43.144492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e1481-6e05-465c-843f-1ee59df11b1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:43.194942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e1481-6e05-465c-843f-1ee59df11b1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:43.196387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e1481-6e05-465c-843f-1ee59df11b1b service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:43.228185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 810/1636] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:43.416140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0951c-f1a1-4caf-8b7e-443bbe19fd94 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:43.431768 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0951c-f1a1-4caf-8b7e-443bbe19fd94 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:43.437055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:43.438198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:43.438484 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:43.438761 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] PATH_INFO: `/v3/users/e65a088d77fd4a55b619c98fa751bb01` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:43.452622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e65a088d77fd4a55b619c98fa751bb01)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:43.453740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: WARNING py.warnings [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:08:43.453740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: warnings.warn(msg) Jul 05 16:08:43.453740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: Jul 05 16:08:43.457271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:43.793765 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:43.795821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e65a088d77fd4a55b619c98fa751bb01'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:43.809300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e65a088d77fd4a55b619c98fa751bb01'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:43.810185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e65a088d77fd4a55b619c98fa751bb01'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:43.810894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e65a088d77fd4a55b619c98fa751bb01'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:43.840628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e65a088d77fd4a55b619c98fa751bb01'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:43.928456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 827/1637] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:08:43 2022] DELETE /identity/v3/users/e65a088d77fd4a55b619c98fa751bb01 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:43.933615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b778bdc-58b9-4fbd-9d8c-fedbd4b68ccc None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:43.934217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b778bdc-58b9-4fbd-9d8c-fedbd4b68ccc None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:43.934217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b778bdc-58b9-4fbd-9d8c-fedbd4b68ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:43.952434 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-6b778bdc-58b9-4fbd-9d8c-fedbd4b68ccc None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:43.956260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-6b778bdc-58b9-4fbd-9d8c-fedbd4b68ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:44.010981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:44.011209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:44.012348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:44.036860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 811/1638] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:44.078744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b778bdc-58b9-4fbd-9d8c-fedbd4b68ccc None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:44.080573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-6b778bdc-58b9-4fbd-9d8c-fedbd4b68ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:44.143345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031196b-f4e1-46aa-9a7a-1731e510d22a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:44.150837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:44.157514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:44.157514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:44.157514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] PATH_INFO: `/v3/users/967d6158df3a4ff696a80d27c1f7483a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:44.157514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=967d6158df3a4ff696a80d27c1f7483a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:44.158192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:44.172758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b778bdc-58b9-4fbd-9d8c-fedbd4b68ccc None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:44.178459 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d5d70-0a87-4580-9d08-da810173d9c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:44.179593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d5d70-0a87-4580-9d08-da810173d9c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:44.180127 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d5d70-0a87-4580-9d08-da810173d9c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:44.187337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d5d70-0a87-4580-9d08-da810173d9c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:44.194676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d5d70-0a87-4580-9d08-da810173d9c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:44.197189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d5d70-0a87-4580-9d08-da810173d9c6 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:44.214759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 812/1639] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:08:44.392666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '967d6158df3a4ff696a80d27c1f7483a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:44.402699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '967d6158df3a4ff696a80d27c1f7483a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:44.403621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '967d6158df3a4ff696a80d27c1f7483a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:44.404438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '967d6158df3a4ff696a80d27c1f7483a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:44.420359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '967d6158df3a4ff696a80d27c1f7483a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:44.498057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 828/1640] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:08:44 2022] DELETE /identity/v3/users/967d6158df3a4ff696a80d27c1f7483a => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:44.502657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdc721-f6c8-40ae-8a98-c340fe4e8b08 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:44.502955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdc721-f6c8-40ae-8a98-c340fe4e8b08 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:44.503237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdc721-f6c8-40ae-8a98-c340fe4e8b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:44.541027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d1fdc721-f6c8-40ae-8a98-c340fe4e8b08 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:44.546741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d1fdc721-f6c8-40ae-8a98-c340fe4e8b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:44.692399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 813/1641] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:44.707675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:44.711043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:44.776361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:44.794244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e2cf1-ce89-4057-ad7d-07a69cee3575 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:44.800320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf74406-9cf2-4550-b867-b628465c35d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:44.802083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf74406-9cf2-4550-b867-b628465c35d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:44.802469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf74406-9cf2-4550-b867-b628465c35d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:44.802860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf74406-9cf2-4550-b867-b628465c35d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:44.815721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf74406-9cf2-4550-b867-b628465c35d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:44.817418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf74406-9cf2-4550-b867-b628465c35d5 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:44.831233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 829/1642] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:45.637357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdc721-f6c8-40ae-8a98-c340fe4e8b08 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:45.642417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d1fdc721-f6c8-40ae-8a98-c340fe4e8b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:45.679097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdc721-f6c8-40ae-8a98-c340fe4e8b08 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:45.684220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:45.686110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:45.686607 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:45.687206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] PATH_INFO: `/v3/projects/986d1a42da2042aba9eff3da2dc9b12e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:45.692023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=986d1a42da2042aba9eff3da2dc9b12e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:45.693835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: WARNING py.warnings [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 16:08:45.693835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: warnings.warn(msg) Jul 05 16:08:45.693835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: Jul 05 16:08:45.695440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:45.866981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986d1a42da2042aba9eff3da2dc9b12e', 'request_id': 'req-0011fb51-438d-4aaf-8946-09136a193bbd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:45.888012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986d1a42da2042aba9eff3da2dc9b12e', 'request_id': 'req-0011fb51-438d-4aaf-8946-09136a193bbd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:46.002656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986d1a42da2042aba9eff3da2dc9b12e', 'request_id': 'req-0011fb51-438d-4aaf-8946-09136a193bbd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:46.049837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] The token cache is being invalidate because project 986d1a42da2042aba9eff3da2dc9b12e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:08:46.050147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:46.051327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0011fb51-438d-4aaf-8946-09136a193bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:46.053273 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 814/1643] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:08:45 2022] DELETE /identity/v3/projects/986d1a42da2042aba9eff3da2dc9b12e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:46.058562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b576d2-20eb-401a-9554-5360ee1a2f2b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:46.059063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b576d2-20eb-401a-9554-5360ee1a2f2b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:46.060182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b576d2-20eb-401a-9554-5360ee1a2f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:46.115445 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-94b576d2-20eb-401a-9554-5360ee1a2f2b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:46.115445 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-94b576d2-20eb-401a-9554-5360ee1a2f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:46.261984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 830/1644] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:46.278280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0bccb-4eba-4b87-8b74-b8ab0a3edd58 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:46.281110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0bccb-4eba-4b87-8b74-b8ab0a3edd58 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:46.281506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0bccb-4eba-4b87-8b74-b8ab0a3edd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:46.315759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e7f0bccb-4eba-4b87-8b74-b8ab0a3edd58 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:46.321765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e7f0bccb-4eba-4b87-8b74-b8ab0a3edd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:46.346898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 815/1645] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:46.349750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b576d2-20eb-401a-9554-5360ee1a2f2b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:46.363372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b576d2-20eb-401a-9554-5360ee1a2f2b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:46.373290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a41626-fd17-4d92-b9f0-16657df4ff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:46.373290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a41626-fd17-4d92-b9f0-16657df4ff7d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:46.373290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a41626-fd17-4d92-b9f0-16657df4ff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:46.373290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a41626-fd17-4d92-b9f0-16657df4ff7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:46.373578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a41626-fd17-4d92-b9f0-16657df4ff7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:46.373578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a41626-fd17-4d92-b9f0-16657df4ff7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:46.374741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a41626-fd17-4d92-b9f0-16657df4ff7d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:46.393868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 831/1646] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:08:46.398176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0bccb-4eba-4b87-8b74-b8ab0a3edd58 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:46.410909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0bccb-4eba-4b87-8b74-b8ab0a3edd58 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:46.423153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b13ce-2fc7-41f0-acbc-d1917b9994a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:46.423987 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b13ce-2fc7-41f0-acbc-d1917b9994a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:46.424692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b13ce-2fc7-41f0-acbc-d1917b9994a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:46.425210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b13ce-2fc7-41f0-acbc-d1917b9994a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:46.426163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b13ce-2fc7-41f0-acbc-d1917b9994a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:46.428554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b13ce-2fc7-41f0-acbc-d1917b9994a6 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:46.498136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:46.577312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a41626-fd17-4d92-b9f0-16657df4ff7d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:46.582700 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-69a41626-fd17-4d92-b9f0-16657df4ff7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:46.595333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 816/1647] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:46 2022] POST /identity/v3/projects => generated 369 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:08:46.599318 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b13ce-2fc7-41f0-acbc-d1917b9994a6 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:46.611248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a41626-fd17-4d92-b9f0-16657df4ff7d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:46.747958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b13ce-2fc7-41f0-acbc-d1917b9994a6 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:46.764636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2676c-42e9-4b77-8440-43d48ca367d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:46.765359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2676c-42e9-4b77-8440-43d48ca367d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:46.765642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2676c-42e9-4b77-8440-43d48ca367d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:46.765939 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2676c-42e9-4b77-8440-43d48ca367d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:46.766610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2676c-42e9-4b77-8440-43d48ca367d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:46.768566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2676c-42e9-4b77-8440-43d48ca367d3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:46.825593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:46.826541 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:46.826827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:46.827114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] PATH_INFO: `/v3/projects/55e522ac286d4dc584d658f850f49d0a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:46.830260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e522ac286d4dc584d658f850f49d0a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:46.832315 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:46.893515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 817/1648] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:46 2022] POST /identity/v3/users => generated 285 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:08:46.894745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2676c-42e9-4b77-8440-43d48ca367d3 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:46.914708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2676c-42e9-4b77-8440-43d48ca367d3 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:46.931740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2c689-261b-4aa8-bd2f-7f67ed6b0b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:46.932495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2c689-261b-4aa8-bd2f-7f67ed6b0b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:46.932701 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2c689-261b-4aa8-bd2f-7f67ed6b0b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:46.933047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2c689-261b-4aa8-bd2f-7f67ed6b0b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:46.933791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2c689-261b-4aa8-bd2f-7f67ed6b0b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:46.943434 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2c689-261b-4aa8-bd2f-7f67ed6b0b23 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:46.954046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 818/1649] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:08:46.960987 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2c689-261b-4aa8-bd2f-7f67ed6b0b23 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:46.991515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2c689-261b-4aa8-bd2f-7f67ed6b0b23 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:46.998404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3040bf-42d8-4c40-8451-9b1e1e0813bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:46.999326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3040bf-42d8-4c40-8451-9b1e1e0813bd admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:46.999665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3040bf-42d8-4c40-8451-9b1e1e0813bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:46.999943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3040bf-42d8-4c40-8451-9b1e1e0813bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:47.000783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3040bf-42d8-4c40-8451-9b1e1e0813bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:47.002607 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3040bf-42d8-4c40-8451-9b1e1e0813bd admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:47.021824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 819/1650] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:47.024146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3040bf-42d8-4c40-8451-9b1e1e0813bd admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:47.055394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3040bf-42d8-4c40-8451-9b1e1e0813bd admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:47.072113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e522ac286d4dc584d658f850f49d0a', 'request_id': 'req-d62f28bd-3041-4e4e-842d-dbb91de0eda9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:47.080862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e522ac286d4dc584d658f850f49d0a', 'request_id': 'req-d62f28bd-3041-4e4e-842d-dbb91de0eda9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:47.092061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e522ac286d4dc584d658f850f49d0a', 'request_id': 'req-d62f28bd-3041-4e4e-842d-dbb91de0eda9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:47.184423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] The token cache is being invalidate because project 55e522ac286d4dc584d658f850f49d0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:08:47.185126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:47.186651 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:47.190796 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 832/1651] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:08:46 2022] DELETE /identity/v3/projects/55e522ac286d4dc584d658f850f49d0a => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:47.195759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:47.199868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:47.205070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501748c5-79f2-4fd2-ba40-0e311db3ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:47.206019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501748c5-79f2-4fd2-ba40-0e311db3ff8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:47.206301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501748c5-79f2-4fd2-ba40-0e311db3ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:47.206675 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501748c5-79f2-4fd2-ba40-0e311db3ff8f admin admin] PATH_INFO: `/v3/projects/27bbdc40ab6a4972a0241ec25a5a9157/users/2b56030f42b3453ca11d0ce758f9163f/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:47.246300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:47.247700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501748c5-79f2-4fd2-ba40-0e311db3ff8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bbdc40ab6a4972a0241ec25a5a9157, user_id=2b56030f42b3453ca11d0ce758f9163f, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:47.250459 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501748c5-79f2-4fd2-ba40-0e311db3ff8f admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:47.264103 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f28bd-3041-4e4e-842d-dbb91de0eda9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:47.276518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:47.277879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:47.278263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:47.278638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] PATH_INFO: `/v3/users/23c522c6cd614eae9738ba4cbca41a3a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:47.280585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c522c6cd614eae9738ba4cbca41a3a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:47.283829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:47.368150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 820/1652] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:47 2022] PUT /identity/v3/projects/27bbdc40ab6a4972a0241ec25a5a9157/users/2b56030f42b3453ca11d0ce758f9163f/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:47.369992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2224c-9fc2-4710-943b-2176ef369533 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:47.377361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2224c-9fc2-4710-943b-2176ef369533 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:47.377729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2224c-9fc2-4710-943b-2176ef369533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:47.452433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f4a2224c-9fc2-4710-943b-2176ef369533 None None] MFA Rules not processed for user `2b56030f42b3453ca11d0ce758f9163f`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:47.453914 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f4a2224c-9fc2-4710-943b-2176ef369533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:47.556954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c522c6cd614eae9738ba4cbca41a3a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:47.582164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c522c6cd614eae9738ba4cbca41a3a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:47.582164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c522c6cd614eae9738ba4cbca41a3a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:47.583030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c522c6cd614eae9738ba4cbca41a3a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:47.620590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c522c6cd614eae9738ba4cbca41a3a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:47.681414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 821/1653] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:47.683638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2224c-9fc2-4710-943b-2176ef369533 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:47.688379 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f4a2224c-9fc2-4710-943b-2176ef369533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:47.717870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 833/1654] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:08:47 2022] DELETE /identity/v3/users/23c522c6cd614eae9738ba4cbca41a3a => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:47.841782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2224c-9fc2-4710-943b-2176ef369533 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:47.971505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ab821-a773-46bf-afbe-11fbd3964f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:47.972185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ab821-a773-46bf-afbe-11fbd3964f76 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:47.972472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ab821-a773-46bf-afbe-11fbd3964f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:47.972858 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ab821-a773-46bf-afbe-11fbd3964f76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:47.982248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ab821-a773-46bf-afbe-11fbd3964f76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:47.986462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ab821-a773-46bf-afbe-11fbd3964f76 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:48.100267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 822/1655] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:47 2022] POST /identity/v3/projects => generated 369 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:48.100529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:48.114596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c00f1-edc5-442a-88b8-95b25593414e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:48.213338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985af30-c81f-40d6-9ef1-90cc01814a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:48.214969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985af30-c81f-40d6-9ef1-90cc01814a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:48.215376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985af30-c81f-40d6-9ef1-90cc01814a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:48.215625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985af30-c81f-40d6-9ef1-90cc01814a1e admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:48.216593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985af30-c81f-40d6-9ef1-90cc01814a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:48.219298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985af30-c81f-40d6-9ef1-90cc01814a1e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:48.382996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:48.396342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 834/1656] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:48 2022] POST /identity/v3/users => generated 291 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:08:48.401094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ab821-a773-46bf-afbe-11fbd3964f76 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:48.411711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ab821-a773-46bf-afbe-11fbd3964f76 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:48.422263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a41c-f670-4948-85f7-cf8347611a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:48.423434 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a41c-f670-4948-85f7-cf8347611a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:48.423868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a41c-f670-4948-85f7-cf8347611a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:48.424355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a41c-f670-4948-85f7-cf8347611a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:48.425533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990a41c-f670-4948-85f7-cf8347611a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:48.427885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990a41c-f670-4948-85f7-cf8347611a83 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:48.442029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 823/1657] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:48.446176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985af30-c81f-40d6-9ef1-90cc01814a1e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:48.460080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985af30-c81f-40d6-9ef1-90cc01814a1e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:48.477287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79a03b-2f92-4988-b211-16ba15a80b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:48.479915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79a03b-2f92-4988-b211-16ba15a80b65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:48.480421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79a03b-2f92-4988-b211-16ba15a80b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:48.480961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79a03b-2f92-4988-b211-16ba15a80b65 admin admin] PATH_INFO: `/v3/projects/575021ca5f494482b1e9ccf99f260aed/users/082925fd7049439192cfa8253442c68d/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:48.506523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79a03b-2f92-4988-b211-16ba15a80b65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=575021ca5f494482b1e9ccf99f260aed, user_id=082925fd7049439192cfa8253442c68d, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:48.510642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79a03b-2f92-4988-b211-16ba15a80b65 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:48.642375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 835/1658] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:48 2022] PUT /identity/v3/projects/575021ca5f494482b1e9ccf99f260aed/users/082925fd7049439192cfa8253442c68d/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:48.646382 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a41c-f670-4948-85f7-cf8347611a83 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:48.660066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a41c-f670-4948-85f7-cf8347611a83 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:48.846123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5524f7-2f3a-4f1e-8215-571a5ef57415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:48.849863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5524f7-2f3a-4f1e-8215-571a5ef57415 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:48.850274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5524f7-2f3a-4f1e-8215-571a5ef57415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:48.850670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5524f7-2f3a-4f1e-8215-571a5ef57415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:48.853845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5524f7-2f3a-4f1e-8215-571a5ef57415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:48.855870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5524f7-2f3a-4f1e-8215-571a5ef57415 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:48.892070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 824/1659] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:48.893071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79a03b-2f92-4988-b211-16ba15a80b65 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:48.906935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79a03b-2f92-4988-b211-16ba15a80b65 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:48.933387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce73f86-bfc0-41cf-b85b-32273cf02814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:48.939610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce73f86-bfc0-41cf-b85b-32273cf02814 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:48.939610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce73f86-bfc0-41cf-b85b-32273cf02814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:48.939610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce73f86-bfc0-41cf-b85b-32273cf02814 admin admin] PATH_INFO: `/v3/projects/575021ca5f494482b1e9ccf99f260aed/users/082925fd7049439192cfa8253442c68d/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:48.940555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce73f86-bfc0-41cf-b85b-32273cf02814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=575021ca5f494482b1e9ccf99f260aed, user_id=082925fd7049439192cfa8253442c68d, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:48.943851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce73f86-bfc0-41cf-b85b-32273cf02814 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:48.991556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 836/1660] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:48 2022] PUT /identity/v3/projects/575021ca5f494482b1e9ccf99f260aed/users/082925fd7049439192cfa8253442c68d/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:08:48.998577 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182f3c2-2142-40c9-8c1c-de5eb0230c0c None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:48.999175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182f3c2-2142-40c9-8c1c-de5eb0230c0c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:48.999657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182f3c2-2142-40c9-8c1c-de5eb0230c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:49.089130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3182f3c2-2142-40c9-8c1c-de5eb0230c0c None None] MFA Rules not processed for user `082925fd7049439192cfa8253442c68d`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:49.090705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3182f3c2-2142-40c9-8c1c-de5eb0230c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:49.273300 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 825/1661] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:49.293038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce73f86-bfc0-41cf-b85b-32273cf02814 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:49.310747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dce73f86-bfc0-41cf-b85b-32273cf02814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:49.336565 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:49.388720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:49.476932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce73f86-bfc0-41cf-b85b-32273cf02814 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:49.493040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c86eb-a0f1-49d3-b263-58c20eb21ff5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:49.493858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c86eb-a0f1-49d3-b263-58c20eb21ff5 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:49.495116 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c86eb-a0f1-49d3-b263-58c20eb21ff5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:49.495472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c86eb-a0f1-49d3-b263-58c20eb21ff5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:49.524176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c86eb-a0f1-49d3-b263-58c20eb21ff5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:49.525814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c86eb-a0f1-49d3-b263-58c20eb21ff5 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:49.551335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 837/1662] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:49 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:49.700041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182f3c2-2142-40c9-8c1c-de5eb0230c0c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:49.701892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3182f3c2-2142-40c9-8c1c-de5eb0230c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:49.729976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182f3c2-2142-40c9-8c1c-de5eb0230c0c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:49.738776 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee9cc2-152e-4aba-9c08-fbe841ae4cd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:49.739763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee9cc2-152e-4aba-9c08-fbe841ae4cd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:49.741122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee9cc2-152e-4aba-9c08-fbe841ae4cd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:49.743840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee9cc2-152e-4aba-9c08-fbe841ae4cd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:49.870909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee9cc2-152e-4aba-9c08-fbe841ae4cd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:49.872365 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee9cc2-152e-4aba-9c08-fbe841ae4cd4 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:49.902085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 826/1663] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:50.369524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c86eb-a0f1-49d3-b263-58c20eb21ff5 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:50.372052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7f0c86eb-a0f1-49d3-b263-58c20eb21ff5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:50.686176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c86eb-a0f1-49d3-b263-58c20eb21ff5 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:50.696239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:50.699120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:50.699411 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:50.699691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] PATH_INFO: `/v3/projects/23c254877bfa424e963e4491565f828f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:50.710906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c254877bfa424e963e4491565f828f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:50.712922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:50.940821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c254877bfa424e963e4491565f828f', 'request_id': 'req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:50.957732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c254877bfa424e963e4491565f828f', 'request_id': 'req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:50.969473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c254877bfa424e963e4491565f828f', 'request_id': 'req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:51.055884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] The token cache is being invalidate because project 23c254877bfa424e963e4491565f828f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:08:51.056355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:51.059844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b858c1ce-a45e-4a3c-a6eb-ef90e32d3169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:51.067390 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 838/1664] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:08:50 2022] DELETE /identity/v3/projects/23c254877bfa424e963e4491565f828f => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:51.073632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fc660-0f3c-4008-bcda-a0b875b1c1d6 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:51.074171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fc660-0f3c-4008-bcda-a0b875b1c1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:51.074629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fc660-0f3c-4008-bcda-a0b875b1c1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:51.126252 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-648fc660-0f3c-4008-bcda-a0b875b1c1d6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:51.127709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-648fc660-0f3c-4008-bcda-a0b875b1c1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:51.300865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 827/1665] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:51.309870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6155756-d809-492f-b415-629eb4636f7f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:51.310831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6155756-d809-492f-b415-629eb4636f7f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:51.311373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6155756-d809-492f-b415-629eb4636f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:51.337325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f6155756-d809-492f-b415-629eb4636f7f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:51.338578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6155756-d809-492f-b415-629eb4636f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:51.361543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 839/1666] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:51.363505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fc660-0f3c-4008-bcda-a0b875b1c1d6 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:51.378854 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fc660-0f3c-4008-bcda-a0b875b1c1d6 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:51.393767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03920cd8-5006-4f26-8b57-5996ebb30daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:51.394631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03920cd8-5006-4f26-8b57-5996ebb30daa admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:51.394911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03920cd8-5006-4f26-8b57-5996ebb30daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:51.395198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03920cd8-5006-4f26-8b57-5996ebb30daa admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:51.395947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03920cd8-5006-4f26-8b57-5996ebb30daa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:51.396259 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03920cd8-5006-4f26-8b57-5996ebb30daa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:51.398573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03920cd8-5006-4f26-8b57-5996ebb30daa admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:51.419745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 828/1667] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:08:51.421857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6155756-d809-492f-b415-629eb4636f7f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:51.446133 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6155756-d809-492f-b415-629eb4636f7f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:51.452639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20323852-9855-4ebf-827b-6d4615b2e4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:51.453491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323852-9855-4ebf-827b-6d4615b2e4cf admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:51.453778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323852-9855-4ebf-827b-6d4615b2e4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:51.454067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323852-9855-4ebf-827b-6d4615b2e4cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:51.455227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20323852-9855-4ebf-827b-6d4615b2e4cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:51.459502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20323852-9855-4ebf-827b-6d4615b2e4cf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:51.637828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 840/1668] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:51 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:51.638985 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03920cd8-5006-4f26-8b57-5996ebb30daa admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:51.645479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:51.652493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03920cd8-5006-4f26-8b57-5996ebb30daa admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:51.833029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee38fc3-cac1-4038-8af6-3712d2ca36b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:51.833932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee38fc3-cac1-4038-8af6-3712d2ca36b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:51.834358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee38fc3-cac1-4038-8af6-3712d2ca36b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:51.834906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee38fc3-cac1-4038-8af6-3712d2ca36b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:51.836706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee38fc3-cac1-4038-8af6-3712d2ca36b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:51.839311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee38fc3-cac1-4038-8af6-3712d2ca36b0 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:51.945404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 829/1669] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:51 2022] POST /identity/v3/users => generated 288 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:08:51.949692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20323852-9855-4ebf-827b-6d4615b2e4cf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:51.973387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20323852-9855-4ebf-827b-6d4615b2e4cf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:51.986665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f87060-5958-4802-bc27-cdcb3732654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:51.987452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f87060-5958-4802-bc27-cdcb3732654b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:51.988095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f87060-5958-4802-bc27-cdcb3732654b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:51.988576 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f87060-5958-4802-bc27-cdcb3732654b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:51.989594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f87060-5958-4802-bc27-cdcb3732654b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:51.998423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f87060-5958-4802-bc27-cdcb3732654b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:52.050078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 841/1670] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:52.053884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee38fc3-cac1-4038-8af6-3712d2ca36b0 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:52.071685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee38fc3-cac1-4038-8af6-3712d2ca36b0 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:52.089959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a49259-c5db-48ee-a279-dfaab5b1daf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:52.097315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a49259-c5db-48ee-a279-dfaab5b1daf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:52.099355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a49259-c5db-48ee-a279-dfaab5b1daf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:52.100570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a49259-c5db-48ee-a279-dfaab5b1daf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:52.102796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a49259-c5db-48ee-a279-dfaab5b1daf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:52.106126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a49259-c5db-48ee-a279-dfaab5b1daf7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:52.142523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 830/1671] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:52.146794 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f87060-5958-4802-bc27-cdcb3732654b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:52.165345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f87060-5958-4802-bc27-cdcb3732654b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:52.171764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff59d5a-b42e-4b36-a4cd-fa7fe814a8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:52.184693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff59d5a-b42e-4b36-a4cd-fa7fe814a8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:52.184693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff59d5a-b42e-4b36-a4cd-fa7fe814a8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:52.184693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff59d5a-b42e-4b36-a4cd-fa7fe814a8fc admin admin] PATH_INFO: `/v3/projects/8195cf24d54d49868ea9786bcdb1ff84/users/a11c71934a5b488f9b933c307d8d4f03/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:52.218682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff59d5a-b42e-4b36-a4cd-fa7fe814a8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8195cf24d54d49868ea9786bcdb1ff84, user_id=a11c71934a5b488f9b933c307d8d4f03, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:52.232623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff59d5a-b42e-4b36-a4cd-fa7fe814a8fc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:52.255436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:52.350334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 842/1672] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:52 2022] PUT /identity/v3/projects/8195cf24d54d49868ea9786bcdb1ff84/users/a11c71934a5b488f9b933c307d8d4f03/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:52.362370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2ff43-b2f0-4b72-b5f6-0079209e3dfd None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:52.362658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2ff43-b2f0-4b72-b5f6-0079209e3dfd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:52.363215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2ff43-b2f0-4b72-b5f6-0079209e3dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:52.450764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4da2ff43-b2f0-4b72-b5f6-0079209e3dfd None None] MFA Rules not processed for user `a11c71934a5b488f9b933c307d8d4f03`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:52.452203 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4da2ff43-b2f0-4b72-b5f6-0079209e3dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:52.630189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 831/1673] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:52.635611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff59d5a-b42e-4b36-a4cd-fa7fe814a8fc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:52.740343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff59d5a-b42e-4b36-a4cd-fa7fe814a8fc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:52.749333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ee350-424c-4fbc-be17-437e903b59c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:52.756829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ee350-424c-4fbc-be17-437e903b59c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:52.757758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ee350-424c-4fbc-be17-437e903b59c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:52.758112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ee350-424c-4fbc-be17-437e903b59c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:52.758950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ee350-424c-4fbc-be17-437e903b59c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:52.763611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ee350-424c-4fbc-be17-437e903b59c3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:52.926485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 843/1674] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:52 2022] POST /identity/v3/projects => generated 377 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:52.930781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2ff43-b2f0-4b72-b5f6-0079209e3dfd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:52.958498 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2ff43-b2f0-4b72-b5f6-0079209e3dfd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:53.109557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271c392-1b19-40c3-858a-49bf064d9553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:53.111189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271c392-1b19-40c3-858a-49bf064d9553 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:53.123442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271c392-1b19-40c3-858a-49bf064d9553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:53.124975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271c392-1b19-40c3-858a-49bf064d9553 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:53.128724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271c392-1b19-40c3-858a-49bf064d9553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:53.133384 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271c392-1b19-40c3-858a-49bf064d9553 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:53.252045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:53.259920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 832/1675] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:52 2022] POST /identity/v3/users => generated 295 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:08:53.264202 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ee350-424c-4fbc-be17-437e903b59c3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:53.291574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ee350-424c-4fbc-be17-437e903b59c3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:53.306166 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bd720-db96-415f-a42e-65fc4ed5c33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:53.306166 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bd720-db96-415f-a42e-65fc4ed5c33a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:53.306166 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bd720-db96-415f-a42e-65fc4ed5c33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:53.306402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bd720-db96-415f-a42e-65fc4ed5c33a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:53.307024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bd720-db96-415f-a42e-65fc4ed5c33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:53.308656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bd720-db96-415f-a42e-65fc4ed5c33a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:53.344079 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 844/1676] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:53.347535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271c392-1b19-40c3-858a-49bf064d9553 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:53.364797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271c392-1b19-40c3-858a-49bf064d9553 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:53.388810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174e498-e238-44c8-9954-61f5198ed754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:53.392972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e498-e238-44c8-9954-61f5198ed754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:53.393261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e498-e238-44c8-9954-61f5198ed754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:53.393544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e498-e238-44c8-9954-61f5198ed754 admin admin] PATH_INFO: `/v3/projects/6b2d4d432c54453793ba02df411012e5/users/d1149191ade74b5eb56d10a94c7d71f6/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:53.453847 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174e498-e238-44c8-9954-61f5198ed754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2d4d432c54453793ba02df411012e5, user_id=d1149191ade74b5eb56d10a94c7d71f6, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:53.456650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174e498-e238-44c8-9954-61f5198ed754 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:53.572296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 833/1677] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:53 2022] PUT /identity/v3/projects/6b2d4d432c54453793ba02df411012e5/users/d1149191ade74b5eb56d10a94c7d71f6/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:53.573337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bd720-db96-415f-a42e-65fc4ed5c33a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:53.605810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bd720-db96-415f-a42e-65fc4ed5c33a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:53.754279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28742a8a-8599-4629-a330-2f73b3d21678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:53.755046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28742a8a-8599-4629-a330-2f73b3d21678 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:53.755255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28742a8a-8599-4629-a330-2f73b3d21678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:53.755537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28742a8a-8599-4629-a330-2f73b3d21678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:53.756273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28742a8a-8599-4629-a330-2f73b3d21678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:53.762597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28742a8a-8599-4629-a330-2f73b3d21678 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:53.796312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 845/1678] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:08:53.800757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174e498-e238-44c8-9954-61f5198ed754 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:53.824311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174e498-e238-44c8-9954-61f5198ed754 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:53.841316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12dd24a-97f3-4998-b62d-58fbfb58a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:53.843364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dd24a-97f3-4998-b62d-58fbfb58a680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:53.844022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dd24a-97f3-4998-b62d-58fbfb58a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:53.844871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dd24a-97f3-4998-b62d-58fbfb58a680 admin admin] PATH_INFO: `/v3/projects/6b2d4d432c54453793ba02df411012e5/users/d1149191ade74b5eb56d10a94c7d71f6/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:53.854642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12dd24a-97f3-4998-b62d-58fbfb58a680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2d4d432c54453793ba02df411012e5, user_id=d1149191ade74b5eb56d10a94c7d71f6, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:53.858712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12dd24a-97f3-4998-b62d-58fbfb58a680 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:53.891367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 834/1679] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:08:53 2022] PUT /identity/v3/projects/6b2d4d432c54453793ba02df411012e5/users/d1149191ade74b5eb56d10a94c7d71f6/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:53.901411 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe38e2b-b721-4ef8-b77f-b5f0d4929858 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:53.901709 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe38e2b-b721-4ef8-b77f-b5f0d4929858 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:53.901995 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe38e2b-b721-4ef8-b77f-b5f0d4929858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:53.964123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0fe38e2b-b721-4ef8-b77f-b5f0d4929858 None None] MFA Rules not processed for user `d1149191ade74b5eb56d10a94c7d71f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:53.965468 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0fe38e2b-b721-4ef8-b77f-b5f0d4929858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:54.119332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 846/1680] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:54.134583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12dd24a-97f3-4998-b62d-58fbfb58a680 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:54.136406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e12dd24a-97f3-4998-b62d-58fbfb58a680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:54.254944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12dd24a-97f3-4998-b62d-58fbfb58a680 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:54.263134 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dcc7a-6b59-4d97-991b-c559e14bb58d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:54.263821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dcc7a-6b59-4d97-991b-c559e14bb58d service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:54.264109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dcc7a-6b59-4d97-991b-c559e14bb58d service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:54.264408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dcc7a-6b59-4d97-991b-c559e14bb58d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:54.268071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:54.395098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dcc7a-6b59-4d97-991b-c559e14bb58d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:54.396584 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dcc7a-6b59-4d97-991b-c559e14bb58d service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:54.422021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 835/1681] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:54.583253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe38e2b-b721-4ef8-b77f-b5f0d4929858 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:54.585517 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0fe38e2b-b721-4ef8-b77f-b5f0d4929858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:54.750012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe38e2b-b721-4ef8-b77f-b5f0d4929858 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:54.759697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:54.760594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:54.760901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:54.761175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] PATH_INFO: `/v3/users/b2497e2db101402e80478eac693d8953` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:54.765791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2497e2db101402e80478eac693d8953)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:54.768044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:55.006739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2497e2db101402e80478eac693d8953'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:55.023686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2497e2db101402e80478eac693d8953'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:55.025501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2497e2db101402e80478eac693d8953'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:55.026834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2497e2db101402e80478eac693d8953'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:55.065228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2497e2db101402e80478eac693d8953'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:55.199478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 847/1682] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:08:54 2022] DELETE /identity/v3/users/b2497e2db101402e80478eac693d8953 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:55.204354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05418179-ba51-47ba-a913-38ae8886eaf9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:55.205734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05418179-ba51-47ba-a913-38ae8886eaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:55.206245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05418179-ba51-47ba-a913-38ae8886eaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:55.245993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-05418179-ba51-47ba-a913-38ae8886eaf9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:55.247257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-05418179-ba51-47ba-a913-38ae8886eaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:55.414403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 836/1683] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:55.466298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:55.469364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:55.612805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a060608-0d42-44f7-a789-dacfeaebc8f9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:55.624080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491236dd-2be7-4f3a-8f16-c5c0c6d23146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:55.624788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491236dd-2be7-4f3a-8f16-c5c0c6d23146 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:55.626216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491236dd-2be7-4f3a-8f16-c5c0c6d23146 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:55.627930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491236dd-2be7-4f3a-8f16-c5c0c6d23146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:55.669402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491236dd-2be7-4f3a-8f16-c5c0c6d23146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:55.673019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491236dd-2be7-4f3a-8f16-c5c0c6d23146 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:55.721518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 848/1684] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:55.831022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9fd3a-df35-4abb-9594-5f37a9061a4c None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:55.831331 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9fd3a-df35-4abb-9594-5f37a9061a4c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:55.831623 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9fd3a-df35-4abb-9594-5f37a9061a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:55.917531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1ff9fd3a-df35-4abb-9594-5f37a9061a4c None None] MFA Rules not processed for user `d7e2fc1ec676438392d7925b43b806f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:55.919598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1ff9fd3a-df35-4abb-9594-5f37a9061a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:55.937980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:56.092444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 837/1685] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:56.111091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491236dd-2be7-4f3a-8f16-c5c0c6d23146 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:56.112635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-491236dd-2be7-4f3a-8f16-c5c0c6d23146 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:56.138555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:56.303062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491236dd-2be7-4f3a-8f16-c5c0c6d23146 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:56.320311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb5d42-204c-47b8-a19b-c4642d04701c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:56.321572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb5d42-204c-47b8-a19b-c4642d04701c service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:56.321944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb5d42-204c-47b8-a19b-c4642d04701c service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:56.322707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb5d42-204c-47b8-a19b-c4642d04701c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:56.365123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb5d42-204c-47b8-a19b-c4642d04701c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:56.367547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb5d42-204c-47b8-a19b-c4642d04701c service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:56.396080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 849/1686] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:57.307859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9fd3a-df35-4abb-9594-5f37a9061a4c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:57.310081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1ff9fd3a-df35-4abb-9594-5f37a9061a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:57.353231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9fd3a-df35-4abb-9594-5f37a9061a4c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:57.398273 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:57.399269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:57.399550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:57.402520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] PATH_INFO: `/v3/users/2b56030f42b3453ca11d0ce758f9163f` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:57.413685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b56030f42b3453ca11d0ce758f9163f)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:57.415706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:57.498728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6c912-192d-48ef-aa8b-f27e03bd89c5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:57.503444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6c912-192d-48ef-aa8b-f27e03bd89c5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:57.503929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6c912-192d-48ef-aa8b-f27e03bd89c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:57.546929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-dfb6c912-192d-48ef-aa8b-f27e03bd89c5 None None] MFA Rules not processed for user `d89990bfd774417ab0eb52b91ec560d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:57.548638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dfb6c912-192d-48ef-aa8b-f27e03bd89c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:57.651559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 850/1687] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:08:57.664521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39087584-da5c-40d3-8d93-af578bf86703 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:57.664838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39087584-da5c-40d3-8d93-af578bf86703 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:57.665118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39087584-da5c-40d3-8d93-af578bf86703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:57.701889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-39087584-da5c-40d3-8d93-af578bf86703 None None] MFA Rules not processed for user `d89990bfd774417ab0eb52b91ec560d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:57.703139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-39087584-da5c-40d3-8d93-af578bf86703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:57.714382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 851/1688] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:57.726239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b56030f42b3453ca11d0ce758f9163f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:57.726495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39087584-da5c-40d3-8d93-af578bf86703 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:57.728316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-39087584-da5c-40d3-8d93-af578bf86703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:57.752164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b56030f42b3453ca11d0ce758f9163f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:57.753265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b56030f42b3453ca11d0ce758f9163f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:57.763720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39087584-da5c-40d3-8d93-af578bf86703 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:57.772867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844b0ba-9d50-40f9-a41f-b359e0ad810e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:57.774201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844b0ba-9d50-40f9-a41f-b359e0ad810e service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:57.774201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844b0ba-9d50-40f9-a41f-b359e0ad810e service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:57.774201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844b0ba-9d50-40f9-a41f-b359e0ad810e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:57.776253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1844b0ba-9d50-40f9-a41f-b359e0ad810e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:57.808656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b56030f42b3453ca11d0ce758f9163f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:57.808804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844b0ba-9d50-40f9-a41f-b359e0ad810e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:57.809701 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b56030f42b3453ca11d0ce758f9163f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:57.811217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844b0ba-9d50-40f9-a41f-b359e0ad810e service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:57.837703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 852/1689] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:08:57.887850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 838/1690] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:08:57 2022] DELETE /identity/v3/users/2b56030f42b3453ca11d0ce758f9163f => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:57.893075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab7382-7bd0-435f-9538-e6c522f254db None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:57.893855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab7382-7bd0-435f-9538-e6c522f254db None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:57.894212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab7382-7bd0-435f-9538-e6c522f254db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:57.947260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b2ab7382-7bd0-435f-9538-e6c522f254db None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:57.948524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b2ab7382-7bd0-435f-9538-e6c522f254db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:58.129359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 853/1691] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:58.165539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:58.167080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:58.295143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681746b3-6c34-4142-9c9e-58bc7e1bbf9d admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:58.299198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830c05b-8a64-4e10-87cb-e5ede9ce9eb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:58.299881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c05b-8a64-4e10-87cb-e5ede9ce9eb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:58.300167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c05b-8a64-4e10-87cb-e5ede9ce9eb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:58.300447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830c05b-8a64-4e10-87cb-e5ede9ce9eb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:58.313039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830c05b-8a64-4e10-87cb-e5ede9ce9eb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:58.315402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830c05b-8a64-4e10-87cb-e5ede9ce9eb6 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:58.334186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 839/1692] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:58.461813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab7382-7bd0-435f-9538-e6c522f254db None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:58.463286 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b2ab7382-7bd0-435f-9538-e6c522f254db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:58.486451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab7382-7bd0-435f-9538-e6c522f254db None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:58.506374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:58.512982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:58.512982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:58.513551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] PATH_INFO: `/v3/projects/aba0375144644b01b15fb971eaf8b184` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:58.515719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba0375144644b01b15fb971eaf8b184)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:58.518311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:58.521855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:58.674858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba0375144644b01b15fb971eaf8b184', 'request_id': 'req-fb394857-1386-49ed-97cb-26c79a31d8d7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:58.696572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba0375144644b01b15fb971eaf8b184', 'request_id': 'req-fb394857-1386-49ed-97cb-26c79a31d8d7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:58.729955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba0375144644b01b15fb971eaf8b184', 'request_id': 'req-fb394857-1386-49ed-97cb-26c79a31d8d7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:58.842384 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] The token cache is being invalidate because project aba0375144644b01b15fb971eaf8b184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:08:58.842688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:58.845256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fb394857-1386-49ed-97cb-26c79a31d8d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:08:58.850827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 854/1693] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:08:58 2022] DELETE /identity/v3/projects/aba0375144644b01b15fb971eaf8b184 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:08:58.852980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91fa7-d2b2-4596-b991-9b05bf15653f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:58.853274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91fa7-d2b2-4596-b991-9b05bf15653f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:58.853558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91fa7-d2b2-4596-b991-9b05bf15653f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:58.905252 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8be91fa7-d2b2-4596-b991-9b05bf15653f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:58.906704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8be91fa7-d2b2-4596-b991-9b05bf15653f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:58.935119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1c44d-6952-4335-adeb-fe9860399385 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:58.935407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1c44d-6952-4335-adeb-fe9860399385 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:58.935701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1c44d-6952-4335-adeb-fe9860399385 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:58.943196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 855/1694] 10.222.0.45 () {54 vars in 912 bytes} [Tue Jul 5 16:08:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:08:58.949249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf1a1a-c872-431b-a171-980e2fa64a04 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:58.949645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf1a1a-c872-431b-a171-980e2fa64a04 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:58.949935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf1a1a-c872-431b-a171-980e2fa64a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:58.972024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:59.073846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cdcf1a1a-c872-431b-a171-980e2fa64a04 None None] MFA Rules not processed for user `887d2ad3cbf7459a8a59e7442ba9744e`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:59.075417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cdcf1a1a-c872-431b-a171-980e2fa64a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:59.096496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 840/1695] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:59.104446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c321041-df00-4880-8b92-c9df979a3833 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:59.105029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c321041-df00-4880-8b92-c9df979a3833 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:59.105756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c321041-df00-4880-8b92-c9df979a3833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:59.150176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8c321041-df00-4880-8b92-c9df979a3833 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:59.151786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8c321041-df00-4880-8b92-c9df979a3833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:59.172588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 841/1696] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:08:59.178165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c321041-df00-4880-8b92-c9df979a3833 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:59.200473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c321041-df00-4880-8b92-c9df979a3833 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:59.206658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6f60e-205b-4906-9e10-b4d673470ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:59.207871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6f60e-205b-4906-9e10-b4d673470ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:59.208535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6f60e-205b-4906-9e10-b4d673470ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:59.209066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6f60e-205b-4906-9e10-b4d673470ac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:59.210063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6f60e-205b-4906-9e10-b4d673470ac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:08:59.211159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6f60e-205b-4906-9e10-b4d673470ac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:59.213062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6f60e-205b-4906-9e10-b4d673470ac1 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:59.227738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 842/1697] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:08:59.232785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6f60e-205b-4906-9e10-b4d673470ac1 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:59.235309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 856/1698] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:08:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:08:59.258119 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6f60e-205b-4906-9e10-b4d673470ac1 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:59.279301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa67b8-5784-4475-bfdf-03851d95e4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:59.279301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa67b8-5784-4475-bfdf-03851d95e4c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:59.279301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa67b8-5784-4475-bfdf-03851d95e4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:59.279301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa67b8-5784-4475-bfdf-03851d95e4c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:59.279963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa67b8-5784-4475-bfdf-03851d95e4c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:59.280912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf1a1a-c872-431b-a171-980e2fa64a04 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:59.282497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cdcf1a1a-c872-431b-a171-980e2fa64a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:59.282727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa67b8-5784-4475-bfdf-03851d95e4c1 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:59.297654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf1a1a-c872-431b-a171-980e2fa64a04 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:59.308707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe80afb-bd96-4de8-9b04-c5a60788d1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:59.309622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80afb-bd96-4de8-9b04-c5a60788d1ee service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:59.310000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80afb-bd96-4de8-9b04-c5a60788d1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:59.310407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80afb-bd96-4de8-9b04-c5a60788d1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:59.325564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe80afb-bd96-4de8-9b04-c5a60788d1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:59.327219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe80afb-bd96-4de8-9b04-c5a60788d1ee service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:59.341555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 857/1699] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:08:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:08:59.401311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 843/1700] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:08:59 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:59.405238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe80afb-bd96-4de8-9b04-c5a60788d1ee service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:08:59.461170 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe80afb-bd96-4de8-9b04-c5a60788d1ee service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:08:59.602794 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8361b-cad3-4fcc-9dfc-c18a82d663a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:08:59.604430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8361b-cad3-4fcc-9dfc-c18a82d663a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:59.604745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8361b-cad3-4fcc-9dfc-c18a82d663a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:59.605045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8361b-cad3-4fcc-9dfc-c18a82d663a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:59.605752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8361b-cad3-4fcc-9dfc-c18a82d663a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:08:59.607688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8361b-cad3-4fcc-9dfc-c18a82d663a7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:08:59.731569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91352f9-ab1a-452e-98aa-17d1fea8599a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:59.736790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91352f9-ab1a-452e-98aa-17d1fea8599a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:59.736790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91352f9-ab1a-452e-98aa-17d1fea8599a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:59.778889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 858/1701] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:08:59 2022] POST /identity/v3/users => generated 286 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:08:59.786214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c91352f9-ab1a-452e-98aa-17d1fea8599a None None] MFA Rules not processed for user `cb0474d4e6dd4d2b9918fd0ebe803dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:59.789959 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af10453-55fb-413c-9266-8e6bf14f7951 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:08:59.790258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af10453-55fb-413c-9266-8e6bf14f7951 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:08:59.790546 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af10453-55fb-413c-9266-8e6bf14f7951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:08:59.794444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c91352f9-ab1a-452e-98aa-17d1fea8599a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:59.814956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:08:59.863067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0af10453-55fb-413c-9266-8e6bf14f7951 None None] MFA Rules not processed for user `8b6e665914774eabba120e986e6bb282`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:08:59.864308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0af10453-55fb-413c-9266-8e6bf14f7951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:08:59.969588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 844/1702] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:08:59.972268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91352f9-ab1a-452e-98aa-17d1fea8599a None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:00.007284 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91352f9-ab1a-452e-98aa-17d1fea8599a None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:00.011144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 859/1703] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:08:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:00.012160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af10453-55fb-413c-9266-8e6bf14f7951 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:00.016915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0af10453-55fb-413c-9266-8e6bf14f7951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:00.020740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e184756-7781-4686-ac39-4f8c938dde83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:00.021678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e184756-7781-4686-ac39-4f8c938dde83 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:00.022133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e184756-7781-4686-ac39-4f8c938dde83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:00.023212 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e184756-7781-4686-ac39-4f8c938dde83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:00.024223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e184756-7781-4686-ac39-4f8c938dde83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:00.027137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e184756-7781-4686-ac39-4f8c938dde83 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:00.056980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 845/1704] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:00.058716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886250e3-b1c1-4869-9920-3281fb6a8302 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:00.060209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886250e3-b1c1-4869-9920-3281fb6a8302 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:00.061393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886250e3-b1c1-4869-9920-3281fb6a8302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:00.101741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-886250e3-b1c1-4869-9920-3281fb6a8302 None None] MFA Rules not processed for user `8b6e665914774eabba120e986e6bb282`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:00.104624 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-886250e3-b1c1-4869-9920-3281fb6a8302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:00.120798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 846/1705] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:00.128718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886250e3-b1c1-4869-9920-3281fb6a8302 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:00.171704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886250e3-b1c1-4869-9920-3281fb6a8302 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:00.173835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b667c-0007-4d16-b572-a202fbc1929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:00.175193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b667c-0007-4d16-b572-a202fbc1929a admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:00.176136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b667c-0007-4d16-b572-a202fbc1929a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:00.176418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b667c-0007-4d16-b572-a202fbc1929a admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:00.179266 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b667c-0007-4d16-b572-a202fbc1929a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:00.186669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b667c-0007-4d16-b572-a202fbc1929a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:00.209158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af10453-55fb-413c-9266-8e6bf14f7951 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:00.214204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 847/1706] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:00.215799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d91f7d-f50f-4c5a-9192-2509b7e31d7e None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:00.216669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d91f7d-f50f-4c5a-9192-2509b7e31d7e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:00.218321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d91f7d-f50f-4c5a-9192-2509b7e31d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:00.224568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea0b06-4300-4d9d-a18f-9e6f73ca040f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:00.225381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea0b06-4300-4d9d-a18f-9e6f73ca040f service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:00.227069 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea0b06-4300-4d9d-a18f-9e6f73ca040f service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:00.227755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea0b06-4300-4d9d-a18f-9e6f73ca040f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:00.252760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea0b06-4300-4d9d-a18f-9e6f73ca040f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:00.254402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea0b06-4300-4d9d-a18f-9e6f73ca040f service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:00.291172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 860/1707] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:00.291172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea0b06-4300-4d9d-a18f-9e6f73ca040f service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:00.298465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-11d91f7d-f50f-4c5a-9192-2509b7e31d7e None None] MFA Rules not processed for user `0c8512e7c1704fac9cc4fcfe3bf1f18b`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:00.299683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-11d91f7d-f50f-4c5a-9192-2509b7e31d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:00.315548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea0b06-4300-4d9d-a18f-9e6f73ca040f service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:00.341579 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594b86c-ebf4-4645-9b4c-f45816b6800b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:00.342421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594b86c-ebf4-4645-9b4c-f45816b6800b admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:00.342704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594b86c-ebf4-4645-9b4c-f45816b6800b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:00.342984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594b86c-ebf4-4645-9b4c-f45816b6800b admin admin] PATH_INFO: `/v3/projects/5f869af7abeb422e9200d378c8b02b78/users/619d9a25f2244269aeaff7b50a8957e4/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:00.402263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594b86c-ebf4-4645-9b4c-f45816b6800b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f869af7abeb422e9200d378c8b02b78, user_id=619d9a25f2244269aeaff7b50a8957e4, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:00.405773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:00.406326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594b86c-ebf4-4645-9b4c-f45816b6800b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:00.459548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 848/1708] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:00.532076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 861/1709] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:00 2022] PUT /identity/v3/projects/5f869af7abeb422e9200d378c8b02b78/users/619d9a25f2244269aeaff7b50a8957e4/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:00.542525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d91f7d-f50f-4c5a-9192-2509b7e31d7e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:00.542525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-11d91f7d-f50f-4c5a-9192-2509b7e31d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:00.552711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cb800-dbf7-43c4-bcd7-8bfcfcb08db1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:00.557391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cb800-dbf7-43c4-bcd7-8bfcfcb08db1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:00.557940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cb800-dbf7-43c4-bcd7-8bfcfcb08db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:00.598159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d91f7d-f50f-4c5a-9192-2509b7e31d7e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:00.668465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0a0cb800-dbf7-43c4-bcd7-8bfcfcb08db1 None None] MFA Rules not processed for user `619d9a25f2244269aeaff7b50a8957e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:00.670421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0a0cb800-dbf7-43c4-bcd7-8bfcfcb08db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:00.853004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:00.855710 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:00.856003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:00.859613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] PATH_INFO: `/v3/projects/27bbdc40ab6a4972a0241ec25a5a9157` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:00.861681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27bbdc40ab6a4972a0241ec25a5a9157)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:00.864599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:00.880933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 862/1710] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:00.882172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cb800-dbf7-43c4-bcd7-8bfcfcb08db1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:00.890241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0a0cb800-dbf7-43c4-bcd7-8bfcfcb08db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:01.052047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cb800-dbf7-43c4-bcd7-8bfcfcb08db1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:01.066130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27bbdc40ab6a4972a0241ec25a5a9157', 'request_id': 'req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:01.074371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27bbdc40ab6a4972a0241ec25a5a9157', 'request_id': 'req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:01.120332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:01.168920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3538d-8ec2-4826-ade7-0346c1460669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:01.170369 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3538d-8ec2-4826-ade7-0346c1460669 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:01.171560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3538d-8ec2-4826-ade7-0346c1460669 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:01.171560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3538d-8ec2-4826-ade7-0346c1460669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:01.173656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-57b3538d-8ec2-4826-ade7-0346c1460669 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:01.178348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27bbdc40ab6a4972a0241ec25a5a9157', 'request_id': 'req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:01.219344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] The token cache is being invalidate because project 27bbdc40ab6a4972a0241ec25a5a9157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:01.219645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:01.223800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:01.240272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 849/1711] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:00 2022] DELETE /identity/v3/projects/27bbdc40ab6a4972a0241ec25a5a9157 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:01.241522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:01.246918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:01.446074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb3e91-63a7-4e5d-a838-b9684e55fd40 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:01.454643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3538d-8ec2-4826-ade7-0346c1460669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:01.456063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3538d-8ec2-4826-ade7-0346c1460669 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:01.460034 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367553b-b8d7-40f5-baa2-30f9b9995ea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:01.461451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367553b-b8d7-40f5-baa2-30f9b9995ea7 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:01.461957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367553b-b8d7-40f5-baa2-30f9b9995ea7 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:01.462435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367553b-b8d7-40f5-baa2-30f9b9995ea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:01.465865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0367553b-b8d7-40f5-baa2-30f9b9995ea7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:01.489360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 863/1712] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:01.491499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3538d-8ec2-4826-ade7-0346c1460669 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:01.496751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-57b3538d-8ec2-4826-ade7-0346c1460669 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:01.541968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:01.548695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3538d-8ec2-4826-ade7-0346c1460669 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:01.561776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd8e71-af7d-471f-b044-46c8cba03344 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:01.562738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd8e71-af7d-471f-b044-46c8cba03344 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:01.563223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd8e71-af7d-471f-b044-46c8cba03344 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:01.563669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd8e71-af7d-471f-b044-46c8cba03344 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:01.571490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a0bd8e71-af7d-471f-b044-46c8cba03344 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:01.681756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367553b-b8d7-40f5-baa2-30f9b9995ea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:01.683423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367553b-b8d7-40f5-baa2-30f9b9995ea7 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:01.708050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 850/1713] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:01.709870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27951a-5937-4e4a-957d-bb259fccd9fd None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:01.710174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27951a-5937-4e4a-957d-bb259fccd9fd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:01.710449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27951a-5937-4e4a-957d-bb259fccd9fd None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:01.711930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 851/1714] 10.222.0.45 () {54 vars in 960 bytes} [Tue Jul 5 16:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:09:01.712643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27951a-5937-4e4a-957d-bb259fccd9fd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:01.722283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-6e27951a-5937-4e4a-957d-bb259fccd9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:01.722363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd8e71-af7d-471f-b044-46c8cba03344 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:01.723948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd8e71-af7d-471f-b044-46c8cba03344 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:01.759426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 864/1715] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:01.761029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0314895-4279-4117-88a0-960564b4547a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:01.765648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0314895-4279-4117-88a0-960564b4547a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:01.765946 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0314895-4279-4117-88a0-960564b4547a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:01.813818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e0314895-4279-4117-88a0-960564b4547a None None] MFA Rules not processed for user `75b7f541993b4a7c915c9e6d440e2674`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:01.815050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e0314895-4279-4117-88a0-960564b4547a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:01.939603 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 865/1716] 10.222.0.45 () {58 vars in 1057 bytes} [Tue Jul 5 16:09:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:01.952741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0314895-4279-4117-88a0-960564b4547a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:01.968576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27951a-5937-4e4a-957d-bb259fccd9fd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:01.974747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0314895-4279-4117-88a0-960564b4547a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:01.981494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:02.001402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424f808-abff-4fa8-9b59-9f4ee9b3409f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:02.002237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:02.002779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:02.003207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424f808-abff-4fa8-9b59-9f4ee9b3409f service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:02.003594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] PATH_INFO: `/v3/users/082925fd7049439192cfa8253442c68d` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:02.004163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424f808-abff-4fa8-9b59-9f4ee9b3409f service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:02.005788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424f808-abff-4fa8-9b59-9f4ee9b3409f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:02.008258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d424f808-abff-4fa8-9b59-9f4ee9b3409f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:02.010345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] RBAC: Authorizing `identity:delete_user(user_id=082925fd7049439192cfa8253442c68d)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:02.014447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:02.025060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424f808-abff-4fa8-9b59-9f4ee9b3409f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:02.027227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424f808-abff-4fa8-9b59-9f4ee9b3409f service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:02.051660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 866/1717] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:02.126661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:02.382061 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '082925fd7049439192cfa8253442c68d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:02.407076 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '082925fd7049439192cfa8253442c68d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:02.408796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '082925fd7049439192cfa8253442c68d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:02.470107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '082925fd7049439192cfa8253442c68d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:02.473474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-52236009-4942-42d2-b50f-a2369fa7b99d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '082925fd7049439192cfa8253442c68d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:02.562095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 852/1718] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:01 2022] DELETE /identity/v3/users/082925fd7049439192cfa8253442c68d => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:02.729591 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424f808-abff-4fa8-9b59-9f4ee9b3409f service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:02.732189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d424f808-abff-4fa8-9b59-9f4ee9b3409f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:02.792356 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec026d46-1c23-4522-ad92-89ffb8aa3aed None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:02.792825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec026d46-1c23-4522-ad92-89ffb8aa3aed None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:02.797023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec026d46-1c23-4522-ad92-89ffb8aa3aed None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:02.797244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424f808-abff-4fa8-9b59-9f4ee9b3409f service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:02.797423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 853/1719] 10.222.0.45 () {54 vars in 912 bytes} [Tue Jul 5 16:09:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:09:02.804541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3e74-7c46-45bf-9439-f827645be94a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:02.804849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3e74-7c46-45bf-9439-f827645be94a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:02.805456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3e74-7c46-45bf-9439-f827645be94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:02.865325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-650e3e74-7c46-45bf-9439-f827645be94a None None] MFA Rules not processed for user `887d2ad3cbf7459a8a59e7442ba9744e`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:02.866944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-650e3e74-7c46-45bf-9439-f827645be94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:02.916878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:03.023877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c0514-cd1e-4259-a5c6-7dda66f86232 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:03.023877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c0514-cd1e-4259-a5c6-7dda66f86232 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:03.024168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c0514-cd1e-4259-a5c6-7dda66f86232 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:03.024395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c0514-cd1e-4259-a5c6-7dda66f86232 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:03.028558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7e2c0514-cd1e-4259-a5c6-7dda66f86232 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:03.065629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 854/1720] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:09:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:03.188539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c0514-cd1e-4259-a5c6-7dda66f86232 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:03.190255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c0514-cd1e-4259-a5c6-7dda66f86232 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:03.212359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 867/1721] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:03.296995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e3e74-7c46-45bf-9439-f827645be94a None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:03.299075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-650e3e74-7c46-45bf-9439-f827645be94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:03.310782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e3e74-7c46-45bf-9439-f827645be94a None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:03.323665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632182a8-2a34-4a35-ae7b-19079de39b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:03.324640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632182a8-2a34-4a35-ae7b-19079de39b82 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:03.325242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632182a8-2a34-4a35-ae7b-19079de39b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:03.325925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632182a8-2a34-4a35-ae7b-19079de39b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:03.347851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632182a8-2a34-4a35-ae7b-19079de39b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:03.352716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632182a8-2a34-4a35-ae7b-19079de39b82 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:03.378379 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 855/1722] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:03.654049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5a7ad-e8e6-461d-a30f-bb70f6e2bd3c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:03.654573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5a7ad-e8e6-461d-a30f-bb70f6e2bd3c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:03.655050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5a7ad-e8e6-461d-a30f-bb70f6e2bd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:03.690930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-94c5a7ad-e8e6-461d-a30f-bb70f6e2bd3c None None] MFA Rules not processed for user `d7e2fc1ec676438392d7925b43b806f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:03.692882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-94c5a7ad-e8e6-461d-a30f-bb70f6e2bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:03.701403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:03.753137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632182a8-2a34-4a35-ae7b-19079de39b82 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:03.755142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-632182a8-2a34-4a35-ae7b-19079de39b82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:03.808228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 868/1723] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:03.932067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632182a8-2a34-4a35-ae7b-19079de39b82 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:03.938125 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:03.939183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:03.939541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:03.939885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] PATH_INFO: `/v3/projects/575021ca5f494482b1e9ccf99f260aed` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:03.941332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=575021ca5f494482b1e9ccf99f260aed)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:03.943424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:04.128693 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575021ca5f494482b1e9ccf99f260aed', 'request_id': 'req-ee45187a-d218-42eb-b185-42fb15614904'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:04.147944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575021ca5f494482b1e9ccf99f260aed', 'request_id': 'req-ee45187a-d218-42eb-b185-42fb15614904'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:04.241168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575021ca5f494482b1e9ccf99f260aed', 'request_id': 'req-ee45187a-d218-42eb-b185-42fb15614904'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:04.259401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] The token cache is being invalidate because project 575021ca5f494482b1e9ccf99f260aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:04.260706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:04.261699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ee45187a-d218-42eb-b185-42fb15614904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:04.265165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 856/1724] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:03 2022] DELETE /identity/v3/projects/575021ca5f494482b1e9ccf99f260aed => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:04.271989 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525a789-1deb-491f-ae36-31ffca2a1982 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:04.277642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525a789-1deb-491f-ae36-31ffca2a1982 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:04.277955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525a789-1deb-491f-ae36-31ffca2a1982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:04.315534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0525a789-1deb-491f-ae36-31ffca2a1982 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:04.317274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0525a789-1deb-491f-ae36-31ffca2a1982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:04.495240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 869/1725] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:04.503047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ea9a7-7a4f-4c17-b6bc-27bb196442b2 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:04.512018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ea9a7-7a4f-4c17-b6bc-27bb196442b2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:04.512946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ea9a7-7a4f-4c17-b6bc-27bb196442b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:04.565165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-6a7ea9a7-7a4f-4c17-b6bc-27bb196442b2 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:04.566483 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-6a7ea9a7-7a4f-4c17-b6bc-27bb196442b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:04.578310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 857/1726] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:04.583037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525a789-1deb-491f-ae36-31ffca2a1982 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:04.602337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525a789-1deb-491f-ae36-31ffca2a1982 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:04.617597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa830d77-d569-434b-9b0b-5183e6734837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:04.618650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830d77-d569-434b-9b0b-5183e6734837 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:04.619333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830d77-d569-434b-9b0b-5183e6734837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:04.619932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830d77-d569-434b-9b0b-5183e6734837 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:04.621031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa830d77-d569-434b-9b0b-5183e6734837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:09:04.621487 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa830d77-d569-434b-9b0b-5183e6734837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:04.623364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa830d77-d569-434b-9b0b-5183e6734837 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:04.644715 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 870/1727] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:09:04.653664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ea9a7-7a4f-4c17-b6bc-27bb196442b2 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:04.668699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ea9a7-7a4f-4c17-b6bc-27bb196442b2 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:04.676312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c171b-9b77-49f0-a700-d7252d497dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:04.677015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c171b-9b77-49f0-a700-d7252d497dac admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:04.678472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c171b-9b77-49f0-a700-d7252d497dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:04.678927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c171b-9b77-49f0-a700-d7252d497dac admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:04.680032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c171b-9b77-49f0-a700-d7252d497dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:04.682844 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c171b-9b77-49f0-a700-d7252d497dac admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:04.777295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa830d77-d569-434b-9b0b-5183e6734837 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:04.778158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 858/1728] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:04 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:04.802325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa830d77-d569-434b-9b0b-5183e6734837 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:04.995007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453ba5e-d253-4d02-add5-3017ec81f643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:04.995676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ba5e-d253-4d02-add5-3017ec81f643 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:04.995961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ba5e-d253-4d02-add5-3017ec81f643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:04.996319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ba5e-d253-4d02-add5-3017ec81f643 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:04.997382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453ba5e-d253-4d02-add5-3017ec81f643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:05.002701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453ba5e-d253-4d02-add5-3017ec81f643 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:05.100691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 871/1729] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:04 2022] POST /identity/v3/users => generated 283 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:09:05.105781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c171b-9b77-49f0-a700-d7252d497dac admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:05.122068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c171b-9b77-49f0-a700-d7252d497dac admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:05.133864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555070f-71d7-4ff4-8bf0-ed80dfad3b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:05.135930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555070f-71d7-4ff4-8bf0-ed80dfad3b20 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:05.135930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555070f-71d7-4ff4-8bf0-ed80dfad3b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:05.137874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555070f-71d7-4ff4-8bf0-ed80dfad3b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:05.142934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555070f-71d7-4ff4-8bf0-ed80dfad3b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:05.163228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555070f-71d7-4ff4-8bf0-ed80dfad3b20 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:05.190856 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 859/1730] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:05.191840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453ba5e-d253-4d02-add5-3017ec81f643 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:05.208384 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2af3c-acb3-4f28-b325-3efe211a34ce None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:05.208671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2af3c-acb3-4f28-b325-3efe211a34ce None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:05.209039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2af3c-acb3-4f28-b325-3efe211a34ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:05.209163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453ba5e-d253-4d02-add5-3017ec81f643 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:05.217090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:05.217957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5944b-0b51-46c2-91fc-e0240712a313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:05.218696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5944b-0b51-46c2-91fc-e0240712a313 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:05.218927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5944b-0b51-46c2-91fc-e0240712a313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:05.219200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5944b-0b51-46c2-91fc-e0240712a313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:05.219931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5944b-0b51-46c2-91fc-e0240712a313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:05.221603 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5944b-0b51-46c2-91fc-e0240712a313 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:05.235139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-83d2af3c-acb3-4f28-b325-3efe211a34ce None None] MFA Rules not processed for user `8b6e665914774eabba120e986e6bb282`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:05.237385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-83d2af3c-acb3-4f28-b325-3efe211a34ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:05.247063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 872/1731] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:05.250594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5944b-0b51-46c2-91fc-e0240712a313 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:05.276672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5944b-0b51-46c2-91fc-e0240712a313 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:05.284129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a2fad-0a05-4768-a09b-3e04cbc2ab38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:05.285391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a2fad-0a05-4768-a09b-3e04cbc2ab38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:05.285829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a2fad-0a05-4768-a09b-3e04cbc2ab38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:05.286237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a2fad-0a05-4768-a09b-3e04cbc2ab38 admin admin] PATH_INFO: `/v3/projects/2fdec9cdf04c4edbb207a0c929b3e85c/users/fada9c05cde04b558610ee368662a6fd/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:05.331504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a2fad-0a05-4768-a09b-3e04cbc2ab38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fdec9cdf04c4edbb207a0c929b3e85c, user_id=fada9c05cde04b558610ee368662a6fd, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:05.334994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a2fad-0a05-4768-a09b-3e04cbc2ab38 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:05.339443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:05.346519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 860/1732] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:05.410806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 873/1733] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:05 2022] PUT /identity/v3/projects/2fdec9cdf04c4edbb207a0c929b3e85c/users/fada9c05cde04b558610ee368662a6fd/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:09:05.423214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27d03-c483-4886-b715-12e3194113f0 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:05.423893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27d03-c483-4886-b715-12e3194113f0 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:05.424320 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27d03-c483-4886-b715-12e3194113f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:05.440320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3d784-8d65-4d37-a7c7-13b72005c095 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:05.444023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3d784-8d65-4d37-a7c7-13b72005c095 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:05.444023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3d784-8d65-4d37-a7c7-13b72005c095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:05.502030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c6f3d784-8d65-4d37-a7c7-13b72005c095 None None] MFA Rules not processed for user `cb0474d4e6dd4d2b9918fd0ebe803dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:05.509087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c6f3d784-8d65-4d37-a7c7-13b72005c095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:05.546584 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-99b27d03-c483-4886-b715-12e3194113f0 None None] MFA Rules not processed for user `fada9c05cde04b558610ee368662a6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:05.547823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-99b27d03-c483-4886-b715-12e3194113f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:05.722861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 874/1734] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:05.776569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 861/1735] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:05.777793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3d784-8d65-4d37-a7c7-13b72005c095 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:06.095207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3d784-8d65-4d37-a7c7-13b72005c095 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:06.115503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bacea-8224-4e58-9f26-47d239c47e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:06.116482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bacea-8224-4e58-9f26-47d239c47e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:06.116935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bacea-8224-4e58-9f26-47d239c47e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:06.123489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bacea-8224-4e58-9f26-47d239c47e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:06.124633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bacea-8224-4e58-9f26-47d239c47e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:06.135457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bacea-8224-4e58-9f26-47d239c47e52 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:06.309214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 875/1736] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:05 2022] POST /identity/v3/projects => generated 365 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:06.310053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b27d03-c483-4886-b715-12e3194113f0 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:06.331908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b27d03-c483-4886-b715-12e3194113f0 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:06.570545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4761-8b98-4540-8995-9ad5da152bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:06.571649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4761-8b98-4540-8995-9ad5da152bec admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:06.572136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4761-8b98-4540-8995-9ad5da152bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:06.572614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4761-8b98-4540-8995-9ad5da152bec admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:06.573981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e4761-8b98-4540-8995-9ad5da152bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:06.579939 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e4761-8b98-4540-8995-9ad5da152bec admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:06.686241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 862/1737] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:06 2022] POST /identity/v3/users => generated 289 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:09:06.689990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bacea-8224-4e58-9f26-47d239c47e52 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:06.710736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bacea-8224-4e58-9f26-47d239c47e52 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:06.726115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daa653-23f2-46d1-ab2c-cea264716037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:06.726115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daa653-23f2-46d1-ab2c-cea264716037 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:06.726115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daa653-23f2-46d1-ab2c-cea264716037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:06.726399 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daa653-23f2-46d1-ab2c-cea264716037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:06.727264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1daa653-23f2-46d1-ab2c-cea264716037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:06.730844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1daa653-23f2-46d1-ab2c-cea264716037 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:06.765021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4761-8b98-4540-8995-9ad5da152bec admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:06.771958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 876/1738] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:06.777017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4761-8b98-4540-8995-9ad5da152bec admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:06.792672 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f8df-6373-4346-bb64-a8af64eb8fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:06.793670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f8df-6373-4346-bb64-a8af64eb8fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:06.794120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f8df-6373-4346-bb64-a8af64eb8fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:06.794543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f8df-6373-4346-bb64-a8af64eb8fb2 admin admin] PATH_INFO: `/v3/projects/d15ee94d59e14a5689a5e0dfe9835943/users/2047e9da33a844a88371ec840dd26a74/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:06.829625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f8df-6373-4346-bb64-a8af64eb8fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15ee94d59e14a5689a5e0dfe9835943, user_id=2047e9da33a844a88371ec840dd26a74, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:06.837130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f8df-6373-4346-bb64-a8af64eb8fb2 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:06.953749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daa653-23f2-46d1-ab2c-cea264716037 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:06.961133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 863/1739] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:06 2022] PUT /identity/v3/projects/d15ee94d59e14a5689a5e0dfe9835943/users/2047e9da33a844a88371ec840dd26a74/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:06.968599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daa653-23f2-46d1-ab2c-cea264716037 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:07.190947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea167212-4907-4b9a-a1ee-bbe968f15ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:07.193038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea167212-4907-4b9a-a1ee-bbe968f15ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:07.193431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea167212-4907-4b9a-a1ee-bbe968f15ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:07.193845 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea167212-4907-4b9a-a1ee-bbe968f15ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:07.194834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea167212-4907-4b9a-a1ee-bbe968f15ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:07.200423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea167212-4907-4b9a-a1ee-bbe968f15ed6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:07.249734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 877/1740] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:07.254474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f8df-6373-4346-bb64-a8af64eb8fb2 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:07.265021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:07.280405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f8df-6373-4346-bb64-a8af64eb8fb2 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:07.306821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355874-9968-4cc9-ba1b-e6984adb799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:07.307737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355874-9968-4cc9-ba1b-e6984adb799b admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:07.308023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355874-9968-4cc9-ba1b-e6984adb799b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:07.308298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355874-9968-4cc9-ba1b-e6984adb799b admin admin] PATH_INFO: `/v3/projects/d15ee94d59e14a5689a5e0dfe9835943/users/2047e9da33a844a88371ec840dd26a74/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:07.310387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46355874-9968-4cc9-ba1b-e6984adb799b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15ee94d59e14a5689a5e0dfe9835943, user_id=2047e9da33a844a88371ec840dd26a74, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:07.322128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46355874-9968-4cc9-ba1b-e6984adb799b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:07.343488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 864/1741] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:07 2022] PUT /identity/v3/projects/d15ee94d59e14a5689a5e0dfe9835943/users/2047e9da33a844a88371ec840dd26a74/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:07.357034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020296a-99c5-41c7-99b2-894caf424444 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:07.357899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020296a-99c5-41c7-99b2-894caf424444 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:07.358279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020296a-99c5-41c7-99b2-894caf424444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:07.415821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:07.441540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9020296a-99c5-41c7-99b2-894caf424444 None None] MFA Rules not processed for user `2047e9da33a844a88371ec840dd26a74`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:07.442957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9020296a-99c5-41c7-99b2-894caf424444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:07.640650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 878/1742] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:07.653389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355874-9968-4cc9-ba1b-e6984adb799b admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:07.655691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-46355874-9968-4cc9-ba1b-e6984adb799b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:07.794615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355874-9968-4cc9-ba1b-e6984adb799b admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:07.803738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32501fc-bee6-4e62-87c9-e4e6006fd842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:07.804882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32501fc-bee6-4e62-87c9-e4e6006fd842 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:07.805407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32501fc-bee6-4e62-87c9-e4e6006fd842 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:07.805822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32501fc-bee6-4e62-87c9-e4e6006fd842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:07.819397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32501fc-bee6-4e62-87c9-e4e6006fd842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:07.822796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32501fc-bee6-4e62-87c9-e4e6006fd842 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:07.841211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 865/1743] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:08.199327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020296a-99c5-41c7-99b2-894caf424444 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:08.213921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020296a-99c5-41c7-99b2-894caf424444 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:08.223173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec577b-b80a-44af-96be-1977d7e29005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:08.223848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec577b-b80a-44af-96be-1977d7e29005 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:08.224139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec577b-b80a-44af-96be-1977d7e29005 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:08.226053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec577b-b80a-44af-96be-1977d7e29005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:08.380531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec577b-b80a-44af-96be-1977d7e29005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:08.382386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec577b-b80a-44af-96be-1977d7e29005 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:08.423506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 879/1744] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:08.524208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9d0e6-a038-4f05-9d15-dd0fc80bb8e3 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:08.525041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9d0e6-a038-4f05-9d15-dd0fc80bb8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:08.525329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9d0e6-a038-4f05-9d15-dd0fc80bb8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:08.631053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e3a9d0e6-a038-4f05-9d15-dd0fc80bb8e3 None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:08.632312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e3a9d0e6-a038-4f05-9d15-dd0fc80bb8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:08.909494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 866/1745] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:09:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:09.565631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec577b-b80a-44af-96be-1977d7e29005 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:09.567159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cdec577b-b80a-44af-96be-1977d7e29005 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:09.805432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec577b-b80a-44af-96be-1977d7e29005 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:09.812488 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c99a2-8f73-409d-98ef-9ae83d7bfda0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:09.813698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c99a2-8f73-409d-98ef-9ae83d7bfda0 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:09.814273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c99a2-8f73-409d-98ef-9ae83d7bfda0 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:09.814745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c99a2-8f73-409d-98ef-9ae83d7bfda0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:09.826063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c99a2-8f73-409d-98ef-9ae83d7bfda0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:09.829281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c99a2-8f73-409d-98ef-9ae83d7bfda0 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:09.844132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 880/1746] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:09:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:09:10.727636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a52d5d-3488-4bf2-89ce-0877d8fb7d0f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:10.728183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a52d5d-3488-4bf2-89ce-0877d8fb7d0f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:10.728501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a52d5d-3488-4bf2-89ce-0877d8fb7d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:10.816337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b4a52d5d-3488-4bf2-89ce-0877d8fb7d0f None None] MFA Rules not processed for user `0c8512e7c1704fac9cc4fcfe3bf1f18b`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:10.822198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b4a52d5d-3488-4bf2-89ce-0877d8fb7d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:11.022443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 867/1747] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:14.706784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c99a2-8f73-409d-98ef-9ae83d7bfda0 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:14.708747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f21c99a2-8f73-409d-98ef-9ae83d7bfda0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:14.814187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:14.925808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c99a2-8f73-409d-98ef-9ae83d7bfda0 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:14.937530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:14.945320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:14.945320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:14.945320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] PATH_INFO: `/v3/users/619d9a25f2244269aeaff7b50a8957e4` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:14.945320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=619d9a25f2244269aeaff7b50a8957e4)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:14.945320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:15.236193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '619d9a25f2244269aeaff7b50a8957e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:15.246006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '619d9a25f2244269aeaff7b50a8957e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:15.246769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '619d9a25f2244269aeaff7b50a8957e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:15.247407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '619d9a25f2244269aeaff7b50a8957e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:15.278271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9398bceb-bbaf-4369-a677-d0d5d7f55cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '619d9a25f2244269aeaff7b50a8957e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:15.291507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a41b7-7831-433c-bed5-f1b34baf1f2f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:15.292262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a41b7-7831-433c-bed5-f1b34baf1f2f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:15.293308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a41b7-7831-433c-bed5-f1b34baf1f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:15.336976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-777a41b7-7831-433c-bed5-f1b34baf1f2f None None] MFA Rules not processed for user `887d2ad3cbf7459a8a59e7442ba9744e`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:15.340479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-777a41b7-7831-433c-bed5-f1b34baf1f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:15.374693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 881/1748] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:14 2022] DELETE /identity/v3/users/619d9a25f2244269aeaff7b50a8957e4 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:15.380609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335a769-c492-415b-825d-3dce0d395925 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:15.381487 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335a769-c492-415b-825d-3dce0d395925 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:15.381755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335a769-c492-415b-825d-3dce0d395925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:15.409636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:15.427424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 868/1749] 10.222.0.47 () {58 vars in 1015 bytes} [Tue Jul 5 16:09:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:15.431892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3335a769-c492-415b-825d-3dce0d395925 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:15.437232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3335a769-c492-415b-825d-3dce0d395925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:15.502924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a41b7-7831-433c-bed5-f1b34baf1f2f None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:15.504398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-777a41b7-7831-433c-bed5-f1b34baf1f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:15.541452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a41b7-7831-433c-bed5-f1b34baf1f2f None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:15.693242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 882/1750] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:15.694818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335a769-c492-415b-825d-3dce0d395925 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:15.718393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2859b2-c886-4a4f-b194-d4c3d3a524d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:15.729818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2859b2-c886-4a4f-b194-d4c3d3a524d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:15.729818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2859b2-c886-4a4f-b194-d4c3d3a524d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:15.730151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2859b2-c886-4a4f-b194-d4c3d3a524d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:15.739957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2859b2-c886-4a4f-b194-d4c3d3a524d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:15.742084 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2859b2-c886-4a4f-b194-d4c3d3a524d5 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:15.779544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 869/1751] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:15.780443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2859b2-c886-4a4f-b194-d4c3d3a524d5 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:15.781833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3d2859b2-c886-4a4f-b194-d4c3d3a524d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:15.794292 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2859b2-c886-4a4f-b194-d4c3d3a524d5 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:15.798809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f749fa1-ddc4-4182-8f6d-b196e5d1999d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:15.799588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f749fa1-ddc4-4182-8f6d-b196e5d1999d service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:15.799875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f749fa1-ddc4-4182-8f6d-b196e5d1999d service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:15.800186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f749fa1-ddc4-4182-8f6d-b196e5d1999d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:15.811778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f749fa1-ddc4-4182-8f6d-b196e5d1999d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:15.822168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f749fa1-ddc4-4182-8f6d-b196e5d1999d service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:15.865201 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 870/1752] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:15.872407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:15.921957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335a769-c492-415b-825d-3dce0d395925 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:15.951651 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291b793-39cd-45ca-a30e-98978e65242f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:15.952339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291b793-39cd-45ca-a30e-98978e65242f service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:15.953832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291b793-39cd-45ca-a30e-98978e65242f service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:15.953832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291b793-39cd-45ca-a30e-98978e65242f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:15.982049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291b793-39cd-45ca-a30e-98978e65242f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:15.983542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291b793-39cd-45ca-a30e-98978e65242f service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:16.033752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 883/1753] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:17.590845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f749fa1-ddc4-4182-8f6d-b196e5d1999d service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:17.593041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8f749fa1-ddc4-4182-8f6d-b196e5d1999d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:17.617061 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f749fa1-ddc4-4182-8f6d-b196e5d1999d service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:17.656213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:17.657389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:17.657822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:17.658278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] PATH_INFO: `/v3/users/fada9c05cde04b558610ee368662a6fd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:17.660644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fada9c05cde04b558610ee368662a6fd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:17.665851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:17.897716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291b793-39cd-45ca-a30e-98978e65242f service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:17.899749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7291b793-39cd-45ca-a30e-98978e65242f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:17.941367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291b793-39cd-45ca-a30e-98978e65242f service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:17.955366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:17.956679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:17.957341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:17.957616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] PATH_INFO: `/v3/projects/5f869af7abeb422e9200d378c8b02b78` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:17.959398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f869af7abeb422e9200d378c8b02b78)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:17.962468 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:18.076970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fada9c05cde04b558610ee368662a6fd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:18.087575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fada9c05cde04b558610ee368662a6fd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:18.088310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fada9c05cde04b558610ee368662a6fd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:18.116358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fada9c05cde04b558610ee368662a6fd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:18.117230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-d1cb7947-dd77-4484-b37c-16b1eed4c70e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fada9c05cde04b558610ee368662a6fd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:18.121801 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f869af7abeb422e9200d378c8b02b78', 'request_id': 'req-3ef21ba3-a96e-4097-993e-addfc32700e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:18.134560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f869af7abeb422e9200d378c8b02b78', 'request_id': 'req-3ef21ba3-a96e-4097-993e-addfc32700e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:18.141858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f869af7abeb422e9200d378c8b02b78', 'request_id': 'req-3ef21ba3-a96e-4097-993e-addfc32700e4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:18.234147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 871/1754] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:17 2022] DELETE /identity/v3/users/fada9c05cde04b558610ee368662a6fd => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:18.235808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff52d6d-2043-45e6-9a40-47c490c61444 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:18.236108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff52d6d-2043-45e6-9a40-47c490c61444 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:18.236393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff52d6d-2043-45e6-9a40-47c490c61444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:18.279596 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-5ff52d6d-2043-45e6-9a40-47c490c61444 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:18.280942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5ff52d6d-2043-45e6-9a40-47c490c61444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:18.324568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] The token cache is being invalidate because project 5f869af7abeb422e9200d378c8b02b78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:18.325201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:18.326097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3ef21ba3-a96e-4097-993e-addfc32700e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:18.340912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 884/1755] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:17 2022] DELETE /identity/v3/projects/5f869af7abeb422e9200d378c8b02b78 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:18.340912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e153d2-a4c1-430c-98b0-2e21ff55bd42 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:18.340912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e153d2-a4c1-430c-98b0-2e21ff55bd42 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:18.340912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e153d2-a4c1-430c-98b0-2e21ff55bd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:18.386541 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-71e153d2-a4c1-430c-98b0-2e21ff55bd42 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:18.397134 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-71e153d2-a4c1-430c-98b0-2e21ff55bd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:18.418643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:18.455351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 872/1756] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:18.457343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff52d6d-2043-45e6-9a40-47c490c61444 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:18.458982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5ff52d6d-2043-45e6-9a40-47c490c61444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:18.576063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff52d6d-2043-45e6-9a40-47c490c61444 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:18.584166 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 885/1757] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:18.584919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dcf03-bf15-4b18-8628-26a89dded62f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:18.585595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dcf03-bf15-4b18-8628-26a89dded62f service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:18.585884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dcf03-bf15-4b18-8628-26a89dded62f service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:18.586166 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dcf03-bf15-4b18-8628-26a89dded62f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:18.592948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0345f28-aff1-48ca-a7de-224a1692cee9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:18.603032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0345f28-aff1-48ca-a7de-224a1692cee9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:18.603342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0345f28-aff1-48ca-a7de-224a1692cee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:18.632053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dcf03-bf15-4b18-8628-26a89dded62f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:18.640246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dcf03-bf15-4b18-8628-26a89dded62f service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:18.667817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 873/1758] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:18.684944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b0345f28-aff1-48ca-a7de-224a1692cee9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:18.690302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b0345f28-aff1-48ca-a7de-224a1692cee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:18.727898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 886/1759] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:18.729363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dcf03-bf15-4b18-8628-26a89dded62f service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:18.748090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dcf03-bf15-4b18-8628-26a89dded62f service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:18.758415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced11d0-a731-4a08-aa5d-063684673f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:18.762541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced11d0-a731-4a08-aa5d-063684673f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:18.762541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced11d0-a731-4a08-aa5d-063684673f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:18.764407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced11d0-a731-4a08-aa5d-063684673f82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:18.765398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced11d0-a731-4a08-aa5d-063684673f82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:09:18.765723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced11d0-a731-4a08-aa5d-063684673f82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:18.778520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced11d0-a731-4a08-aa5d-063684673f82 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:18.797169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 874/1760] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:09:18.801653 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0345f28-aff1-48ca-a7de-224a1692cee9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:18.817716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0345f28-aff1-48ca-a7de-224a1692cee9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:18.831114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47a958-6d2e-45cb-b110-f7db8866b677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:18.833688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47a958-6d2e-45cb-b110-f7db8866b677 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:18.834575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47a958-6d2e-45cb-b110-f7db8866b677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:18.835027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47a958-6d2e-45cb-b110-f7db8866b677 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:18.839411 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47a958-6d2e-45cb-b110-f7db8866b677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:18.843307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47a958-6d2e-45cb-b110-f7db8866b677 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:18.979503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 887/1761] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:18 2022] POST /identity/v3/projects => generated 405 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:18.983190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced11d0-a731-4a08-aa5d-063684673f82 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:19.035365 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced11d0-a731-4a08-aa5d-063684673f82 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:19.206199 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e92292-a92f-4201-8ee5-ef8b8f5c13b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:19.206933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e92292-a92f-4201-8ee5-ef8b8f5c13b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:19.207224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e92292-a92f-4201-8ee5-ef8b8f5c13b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:19.208790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e92292-a92f-4201-8ee5-ef8b8f5c13b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:19.210075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e92292-a92f-4201-8ee5-ef8b8f5c13b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:19.212247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e92292-a92f-4201-8ee5-ef8b8f5c13b5 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:19.312611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 875/1762] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:18 2022] POST /identity/v3/users => generated 303 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:19.312822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47a958-6d2e-45cb-b110-f7db8866b677 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:19.324525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47a958-6d2e-45cb-b110-f7db8866b677 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:19.331671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc6728-3035-4d10-9bed-4b1483107b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:19.332363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc6728-3035-4d10-9bed-4b1483107b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:19.332678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc6728-3035-4d10-9bed-4b1483107b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:19.332957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc6728-3035-4d10-9bed-4b1483107b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:19.338738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc6728-3035-4d10-9bed-4b1483107b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:19.340368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc6728-3035-4d10-9bed-4b1483107b04 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:19.368627 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 888/1763] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:19.372947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e92292-a92f-4201-8ee5-ef8b8f5c13b5 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:19.385738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e92292-a92f-4201-8ee5-ef8b8f5c13b5 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:19.394552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e2650-7c31-43f4-a99b-e2ca2c9c690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:19.397913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e2650-7c31-43f4-a99b-e2ca2c9c690a admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:19.398400 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e2650-7c31-43f4-a99b-e2ca2c9c690a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:19.398925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e2650-7c31-43f4-a99b-e2ca2c9c690a admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:19.399869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e2650-7c31-43f4-a99b-e2ca2c9c690a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:19.401494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e2650-7c31-43f4-a99b-e2ca2c9c690a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:19.417080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 876/1764] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:19.420543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc6728-3035-4d10-9bed-4b1483107b04 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:19.436172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b98b8-68ce-48a9-926c-146c56dac4a4 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:19.437025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b98b8-68ce-48a9-926c-146c56dac4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:19.439592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b98b8-68ce-48a9-926c-146c56dac4a4 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:19.444051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 877/1765] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:09:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:09:19.449286 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc6728-3035-4d10-9bed-4b1483107b04 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:19.463124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcabaef-e9e3-48fa-813c-ecf9d9c2c2bd None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:19.463321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcabaef-e9e3-48fa-813c-ecf9d9c2c2bd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:19.463589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcabaef-e9e3-48fa-813c-ecf9d9c2c2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:19.471431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42addf11-f133-4cd1-8a22-726197364cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:19.472518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42addf11-f133-4cd1-8a22-726197364cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:19.472953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42addf11-f133-4cd1-8a22-726197364cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:19.481675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42addf11-f133-4cd1-8a22-726197364cb1 admin admin] PATH_INFO: `/v3/projects/5c373d86255d484a8e1cfe1f8dca8b8f/users/c084f1a10a6f4dd48cfb78587cd97ed2/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:19.523695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4fcabaef-e9e3-48fa-813c-ecf9d9c2c2bd None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:19.525230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4fcabaef-e9e3-48fa-813c-ecf9d9c2c2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:19.529308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42addf11-f133-4cd1-8a22-726197364cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c373d86255d484a8e1cfe1f8dca8b8f, user_id=c084f1a10a6f4dd48cfb78587cd97ed2, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:19.532158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42addf11-f133-4cd1-8a22-726197364cb1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:19.642711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:19.654465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 889/1766] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:19 2022] PUT /identity/v3/projects/5c373d86255d484a8e1cfe1f8dca8b8f/users/c084f1a10a6f4dd48cfb78587cd97ed2/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:19.664927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6300d-f3e5-4f8b-a4ab-b3776149fadb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:19.665290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6300d-f3e5-4f8b-a4ab-b3776149fadb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:19.665410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 878/1767] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:09:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:19.665564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6300d-f3e5-4f8b-a4ab-b3776149fadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:19.677823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:19.686233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcabaef-e9e3-48fa-813c-ecf9d9c2c2bd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:19.688302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4fcabaef-e9e3-48fa-813c-ecf9d9c2c2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:19.707168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d7d6300d-f3e5-4f8b-a4ab-b3776149fadb None None] MFA Rules not processed for user `c084f1a10a6f4dd48cfb78587cd97ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:19.713797 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d7d6300d-f3e5-4f8b-a4ab-b3776149fadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:19.719725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcabaef-e9e3-48fa-813c-ecf9d9c2c2bd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:19.810935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 890/1768] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:19 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:09:19.811648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6300d-f3e5-4f8b-a4ab-b3776149fadb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:19.812986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d7d6300d-f3e5-4f8b-a4ab-b3776149fadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:19.820678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a772d5f-db7e-4416-8ba2-d6e1147ffd0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:19.821302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a772d5f-db7e-4416-8ba2-d6e1147ffd0c service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:19.821790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a772d5f-db7e-4416-8ba2-d6e1147ffd0c service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:19.821899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a772d5f-db7e-4416-8ba2-d6e1147ffd0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:19.835620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a772d5f-db7e-4416-8ba2-d6e1147ffd0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:19.837272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a772d5f-db7e-4416-8ba2-d6e1147ffd0c service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:19.850840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 879/1769] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:19.851478 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a772d5f-db7e-4416-8ba2-d6e1147ffd0c service nova] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:19.930616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6300d-f3e5-4f8b-a4ab-b3776149fadb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:19.938473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:19.939357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:19.939757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:19.940043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] PATH_INFO: `/v3/projects/2fdec9cdf04c4edbb207a0c929b3e85c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:19.947949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fdec9cdf04c4edbb207a0c929b3e85c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:19.947949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:19.958168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a772d5f-db7e-4416-8ba2-d6e1147ffd0c service nova] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:19.966347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b43217-cae8-4fda-8a64-47ee91763ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:19.967180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b43217-cae8-4fda-8a64-47ee91763ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:19.967570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b43217-cae8-4fda-8a64-47ee91763ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:19.967953 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b43217-cae8-4fda-8a64-47ee91763ee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:19.968983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b43217-cae8-4fda-8a64-47ee91763ee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:19.971491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b43217-cae8-4fda-8a64-47ee91763ee0 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:20.060288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 880/1770] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:19 2022] POST /identity/v3/projects => generated 403 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:20.064568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b43217-cae8-4fda-8a64-47ee91763ee0 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:20.142319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fdec9cdf04c4edbb207a0c929b3e85c', 'request_id': 'req-b4e1a3e9-db57-422c-8493-b8215a44f650'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:20.150363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fdec9cdf04c4edbb207a0c929b3e85c', 'request_id': 'req-b4e1a3e9-db57-422c-8493-b8215a44f650'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:20.153443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b43217-cae8-4fda-8a64-47ee91763ee0 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:20.165971 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fdec9cdf04c4edbb207a0c929b3e85c', 'request_id': 'req-b4e1a3e9-db57-422c-8493-b8215a44f650'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:20.167352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:20.238025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24234029-2dcc-4095-9ed3-9747eb57789b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:20.239052 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24234029-2dcc-4095-9ed3-9747eb57789b admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:20.239301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24234029-2dcc-4095-9ed3-9747eb57789b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:20.239617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24234029-2dcc-4095-9ed3-9747eb57789b admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:20.240432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24234029-2dcc-4095-9ed3-9747eb57789b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:20.243741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24234029-2dcc-4095-9ed3-9747eb57789b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:20.247380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] The token cache is being invalidate because project 2fdec9cdf04c4edbb207a0c929b3e85c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:20.248541 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:20.249595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:20.251932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 891/1771] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:19 2022] DELETE /identity/v3/projects/2fdec9cdf04c4edbb207a0c929b3e85c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:20.257050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:20.258574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:20.272698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1a3e9-db57-422c-8493-b8215a44f650 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:20.280374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:20.281311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:20.281595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:20.281876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] PATH_INFO: `/v3/users/2047e9da33a844a88371ec840dd26a74` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:20.283187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2047e9da33a844a88371ec840dd26a74)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:20.285499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:20.320268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:20.381935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 881/1772] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:20 2022] POST /identity/v3/users => generated 308 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:20.383338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24234029-2dcc-4095-9ed3-9747eb57789b admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:20.386069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-24234029-2dcc-4095-9ed3-9747eb57789b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:20.407179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24234029-2dcc-4095-9ed3-9747eb57789b admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:20.413436 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e449e-dcfe-4eaa-9e06-b45d9d41566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:20.414109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e449e-dcfe-4eaa-9e06-b45d9d41566c admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:20.414404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e449e-dcfe-4eaa-9e06-b45d9d41566c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:20.414684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e449e-dcfe-4eaa-9e06-b45d9d41566c admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:20.415416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e449e-dcfe-4eaa-9e06-b45d9d41566c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:20.417085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e449e-dcfe-4eaa-9e06-b45d9d41566c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:20.429257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 882/1773] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:09:20.432969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e449e-dcfe-4eaa-9e06-b45d9d41566c admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:20.443660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e449e-dcfe-4eaa-9e06-b45d9d41566c admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:20.449007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3734f17-a5b5-4809-8f98-f0ba471157af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:20.450419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3734f17-a5b5-4809-8f98-f0ba471157af admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:20.450614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3734f17-a5b5-4809-8f98-f0ba471157af admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:20.450899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3734f17-a5b5-4809-8f98-f0ba471157af admin admin] PATH_INFO: `/v3/projects/1d4ba2819fc84445a2592208b034b9fb/users/bd0d66bd68454ea196bf51de5c9c4f82/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:20.470106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3734f17-a5b5-4809-8f98-f0ba471157af admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4ba2819fc84445a2592208b034b9fb, user_id=bd0d66bd68454ea196bf51de5c9c4f82, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:20.472879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3734f17-a5b5-4809-8f98-f0ba471157af admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:20.514592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2047e9da33a844a88371ec840dd26a74'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:20.538308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2047e9da33a844a88371ec840dd26a74'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:20.539607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2047e9da33a844a88371ec840dd26a74'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:20.540211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2047e9da33a844a88371ec840dd26a74'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:20.559235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2047e9da33a844a88371ec840dd26a74'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:20.595602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 883/1774] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:20 2022] PUT /identity/v3/projects/1d4ba2819fc84445a2592208b034b9fb/users/bd0d66bd68454ea196bf51de5c9c4f82/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:09:20.599263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3734f17-a5b5-4809-8f98-f0ba471157af admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:20.602643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a3734f17-a5b5-4809-8f98-f0ba471157af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:20.668294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 892/1775] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:20 2022] DELETE /identity/v3/users/2047e9da33a844a88371ec840dd26a74 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:20.714995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3734f17-a5b5-4809-8f98-f0ba471157af admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:20.801253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d20d9-2f4b-479c-86e0-dadfeaa5ddbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:20.801938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d20d9-2f4b-479c-86e0-dadfeaa5ddbf admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:20.802227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d20d9-2f4b-479c-86e0-dadfeaa5ddbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:20.802529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d20d9-2f4b-479c-86e0-dadfeaa5ddbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:20.803299 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d20d9-2f4b-479c-86e0-dadfeaa5ddbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:20.805109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d20d9-2f4b-479c-86e0-dadfeaa5ddbf admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:20.815655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 884/1776] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:09:20.820103 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:20.835363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c436621-d3bc-42f8-94c3-38ca85f6af62 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:20.839976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3890af-f28e-47dd-8610-44f6a75c3f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:20.840912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3890af-f28e-47dd-8610-44f6a75c3f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:20.841199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3890af-f28e-47dd-8610-44f6a75c3f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:20.841475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3890af-f28e-47dd-8610-44f6a75c3f40 admin admin] PATH_INFO: `/v3/projects/1d4ba2819fc84445a2592208b034b9fb/users/bd0d66bd68454ea196bf51de5c9c4f82/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:20.843766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3890af-f28e-47dd-8610-44f6a75c3f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4ba2819fc84445a2592208b034b9fb, user_id=bd0d66bd68454ea196bf51de5c9c4f82, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:20.846469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3890af-f28e-47dd-8610-44f6a75c3f40 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:20.867877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 893/1777] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:20 2022] PUT /identity/v3/projects/1d4ba2819fc84445a2592208b034b9fb/users/bd0d66bd68454ea196bf51de5c9c4f82/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:09:20.878035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c3cb-620d-4b70-814c-53366a3e1001 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:20.878230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c3cb-620d-4b70-814c-53366a3e1001 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:20.878683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3c3cb-620d-4b70-814c-53366a3e1001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:20.914654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:20.924252 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-2bf3c3cb-620d-4b70-814c-53366a3e1001 None None] MFA Rules not processed for user `bd0d66bd68454ea196bf51de5c9c4f82`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:20.925656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2bf3c3cb-620d-4b70-814c-53366a3e1001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:21.009089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 885/1778] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:09:21.027565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3890af-f28e-47dd-8610-44f6a75c3f40 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:21.029366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0f3890af-f28e-47dd-8610-44f6a75c3f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:21.148058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3890af-f28e-47dd-8610-44f6a75c3f40 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:21.158031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc75b3d-31b4-4c40-bf6f-774d9d5e437b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:21.158818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc75b3d-31b4-4c40-bf6f-774d9d5e437b service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:21.159178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc75b3d-31b4-4c40-bf6f-774d9d5e437b service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:21.159534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc75b3d-31b4-4c40-bf6f-774d9d5e437b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:21.175505 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc75b3d-31b4-4c40-bf6f-774d9d5e437b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:21.177592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc75b3d-31b4-4c40-bf6f-774d9d5e437b service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:21.189901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 894/1779] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:21 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:21.242408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3c3cb-620d-4b70-814c-53366a3e1001 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:21.245029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2bf3c3cb-620d-4b70-814c-53366a3e1001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:21.419007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc75b3d-31b4-4c40-bf6f-774d9d5e437b service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:21.424215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3c3cb-620d-4b70-814c-53366a3e1001 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:21.429938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:21.431713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:21.431713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:21.431713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] PATH_INFO: `/v3/projects/d15ee94d59e14a5689a5e0dfe9835943` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:21.433010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d15ee94d59e14a5689a5e0dfe9835943)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:21.435105 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:21.437538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc75b3d-31b4-4c40-bf6f-774d9d5e437b service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:21.443967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:21.444893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:21.445185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:21.445463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] PATH_INFO: `/v3/users/c084f1a10a6f4dd48cfb78587cd97ed2` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:21.447395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c084f1a10a6f4dd48cfb78587cd97ed2)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:21.449913 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:21.657625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c084f1a10a6f4dd48cfb78587cd97ed2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:21.674642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd15ee94d59e14a5689a5e0dfe9835943', 'request_id': 'req-55c84d26-f5b4-44fb-b410-3de7f624c301'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:21.681151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c084f1a10a6f4dd48cfb78587cd97ed2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:21.683774 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c084f1a10a6f4dd48cfb78587cd97ed2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:21.685928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd15ee94d59e14a5689a5e0dfe9835943', 'request_id': 'req-55c84d26-f5b4-44fb-b410-3de7f624c301'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:21.687957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c084f1a10a6f4dd48cfb78587cd97ed2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:21.722756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-299e8889-0aba-4a97-bd44-de77e4d2ae00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c084f1a10a6f4dd48cfb78587cd97ed2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:21.838244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd15ee94d59e14a5689a5e0dfe9835943', 'request_id': 'req-55c84d26-f5b4-44fb-b410-3de7f624c301'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:21.851632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] The token cache is being invalidate because project d15ee94d59e14a5689a5e0dfe9835943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:21.851927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:21.852584 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-55c84d26-f5b4-44fb-b410-3de7f624c301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:21.854205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 886/1780] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:21 2022] DELETE /identity/v3/projects/d15ee94d59e14a5689a5e0dfe9835943 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:21.859735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d102b-ccf6-445d-9218-a7add7129ea8 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:21.860015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d102b-ccf6-445d-9218-a7add7129ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:21.860372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d102b-ccf6-445d-9218-a7add7129ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:21.880168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b35d102b-ccf6-445d-9218-a7add7129ea8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:21.881737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b35d102b-ccf6-445d-9218-a7add7129ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:21.906078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:21.949867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 887/1781] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:21.956790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:21.957848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0e869-a80d-49bd-94ce-4d878594420f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:21.958134 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0e869-a80d-49bd-94ce-4d878594420f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:21.958419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0e869-a80d-49bd-94ce-4d878594420f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:21.961847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 895/1782] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:21 2022] DELETE /identity/v3/users/c084f1a10a6f4dd48cfb78587cd97ed2 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:21.966052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb793c6-022f-4da7-885e-7b3ea7332b36 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:21.966402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb793c6-022f-4da7-885e-7b3ea7332b36 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:21.966689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb793c6-022f-4da7-885e-7b3ea7332b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:21.978339 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-27c0e869-a80d-49bd-94ce-4d878594420f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:21.979653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-27c0e869-a80d-49bd-94ce-4d878594420f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:22.003631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bbb793c6-022f-4da7-885e-7b3ea7332b36 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:22.005177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bbb793c6-022f-4da7-885e-7b3ea7332b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:22.105548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 888/1783] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:09:22.106568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0e869-a80d-49bd-94ce-4d878594420f None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:22.120285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 896/1784] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:09:22.123950 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0e869-a80d-49bd-94ce-4d878594420f None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:22.127876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb793c6-022f-4da7-885e-7b3ea7332b36 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:22.129633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bbb793c6-022f-4da7-885e-7b3ea7332b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:22.133591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d81cc8-7135-4b97-a8f5-312f469ff302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:22.134664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d81cc8-7135-4b97-a8f5-312f469ff302 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:22.135129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d81cc8-7135-4b97-a8f5-312f469ff302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:22.135585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d81cc8-7135-4b97-a8f5-312f469ff302 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:22.136631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d81cc8-7135-4b97-a8f5-312f469ff302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:09:22.136977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d81cc8-7135-4b97-a8f5-312f469ff302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:22.138628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d81cc8-7135-4b97-a8f5-312f469ff302 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:22.152823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 889/1785] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:09:22.158341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d81cc8-7135-4b97-a8f5-312f469ff302 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:22.172265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d81cc8-7135-4b97-a8f5-312f469ff302 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:22.179524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c4bd2-7df7-4208-a5b2-bd7b81465395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:22.180477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c4bd2-7df7-4208-a5b2-bd7b81465395 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:22.180967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c4bd2-7df7-4208-a5b2-bd7b81465395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:22.181604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c4bd2-7df7-4208-a5b2-bd7b81465395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:22.182656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c4bd2-7df7-4208-a5b2-bd7b81465395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:22.184875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c4bd2-7df7-4208-a5b2-bd7b81465395 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:22.230171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb793c6-022f-4da7-885e-7b3ea7332b36 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:22.234400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760dcfa5-22ca-44ae-83e3-2c81e57ddac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:22.235057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dcfa5-22ca-44ae-83e3-2c81e57ddac4 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:22.235471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dcfa5-22ca-44ae-83e3-2c81e57ddac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:22.235749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dcfa5-22ca-44ae-83e3-2c81e57ddac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:22.246914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760dcfa5-22ca-44ae-83e3-2c81e57ddac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:22.249491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760dcfa5-22ca-44ae-83e3-2c81e57ddac4 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:22.273698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 897/1786] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:22.342232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 890/1787] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:22 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:22.354038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760dcfa5-22ca-44ae-83e3-2c81e57ddac4 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:22.380013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760dcfa5-22ca-44ae-83e3-2c81e57ddac4 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:22.457631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb446bb4-9a21-49a2-9c7c-175704f44718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:22.458409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb446bb4-9a21-49a2-9c7c-175704f44718 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:22.459549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb446bb4-9a21-49a2-9c7c-175704f44718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:22.459955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb446bb4-9a21-49a2-9c7c-175704f44718 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:22.461010 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb446bb4-9a21-49a2-9c7c-175704f44718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:22.463482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb446bb4-9a21-49a2-9c7c-175704f44718 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:22.582778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 898/1788] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:22 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:22.592555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c4bd2-7df7-4208-a5b2-bd7b81465395 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:22.608389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c4bd2-7df7-4208-a5b2-bd7b81465395 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:22.617962 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ea867-57c1-4a3a-9461-dd0d70f04b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:22.641491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ea867-57c1-4a3a-9461-dd0d70f04b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:22.641952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ea867-57c1-4a3a-9461-dd0d70f04b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:22.642209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ea867-57c1-4a3a-9461-dd0d70f04b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:22.643171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ea867-57c1-4a3a-9461-dd0d70f04b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:22.653956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ea867-57c1-4a3a-9461-dd0d70f04b66 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:22.706100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 891/1789] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:22.707919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb446bb4-9a21-49a2-9c7c-175704f44718 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:22.724892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb446bb4-9a21-49a2-9c7c-175704f44718 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:22.758936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf8668-9bac-416a-aaea-2450127c4d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:22.759658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf8668-9bac-416a-aaea-2450127c4d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:22.759976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf8668-9bac-416a-aaea-2450127c4d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:22.760290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf8668-9bac-416a-aaea-2450127c4d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:22.761056 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf8668-9bac-416a-aaea-2450127c4d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:22.762676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf8668-9bac-416a-aaea-2450127c4d10 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:22.783018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 899/1790] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:22.787204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ea867-57c1-4a3a-9461-dd0d70f04b66 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:22.797958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ea867-57c1-4a3a-9461-dd0d70f04b66 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:22.804327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bf70e-c780-4926-baab-8dfdd62a8335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:22.805697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf70e-c780-4926-baab-8dfdd62a8335 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:22.806243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf70e-c780-4926-baab-8dfdd62a8335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:22.806551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf70e-c780-4926-baab-8dfdd62a8335 admin admin] PATH_INFO: `/v3/projects/bbaa897514934520bd6eb61aaa872290/users/66375274e9fb44d39936ebf64af69152/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:22.828418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bf70e-c780-4926-baab-8dfdd62a8335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbaa897514934520bd6eb61aaa872290, user_id=66375274e9fb44d39936ebf64af69152, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:22.832715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bf70e-c780-4926-baab-8dfdd62a8335 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:22.868645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf8668-9bac-416a-aaea-2450127c4d10 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:22.870408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-64bf8668-9bac-416a-aaea-2450127c4d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:22.895211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf8668-9bac-416a-aaea-2450127c4d10 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:22.906481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:22.906481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:22.906481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:22.906481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] PATH_INFO: `/v3/projects/5c373d86255d484a8e1cfe1f8dca8b8f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:22.906813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c373d86255d484a8e1cfe1f8dca8b8f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:22.906813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:22.935370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 892/1791] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:22 2022] PUT /identity/v3/projects/bbaa897514934520bd6eb61aaa872290/users/66375274e9fb44d39936ebf64af69152/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:22.940063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7f6ad-c8e8-443e-be1d-deb7033acf46 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:22.940407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7f6ad-c8e8-443e-be1d-deb7033acf46 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:22.940868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7f6ad-c8e8-443e-be1d-deb7033acf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:22.949078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:22.970617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-80c7f6ad-c8e8-443e-be1d-deb7033acf46 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:22.971915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-80c7f6ad-c8e8-443e-be1d-deb7033acf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:23.063212 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:23.072898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c373d86255d484a8e1cfe1f8dca8b8f', 'request_id': 'req-44bf2000-c942-48de-b4b9-affbf65a30f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:23.091174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 893/1792] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:23.092077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c373d86255d484a8e1cfe1f8dca8b8f', 'request_id': 'req-44bf2000-c942-48de-b4b9-affbf65a30f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:23.102015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7f6ad-c8e8-443e-be1d-deb7033acf46 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:23.103621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-80c7f6ad-c8e8-443e-be1d-deb7033acf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:23.106247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c373d86255d484a8e1cfe1f8dca8b8f', 'request_id': 'req-44bf2000-c942-48de-b4b9-affbf65a30f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:23.188827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] The token cache is being invalidate because project 5c373d86255d484a8e1cfe1f8dca8b8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:23.189197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:23.190829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:23.200850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 900/1793] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:22 2022] DELETE /identity/v3/projects/5c373d86255d484a8e1cfe1f8dca8b8f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:23.207780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:23.211909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:23.230583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7f6ad-c8e8-443e-be1d-deb7033acf46 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:23.347237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf2000-c942-48de-b4b9-affbf65a30f5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:23.357672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:23.366427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:23.367213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:23.367573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] PATH_INFO: `/v3/users/bd0d66bd68454ea196bf51de5c9c4f82` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:23.368513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc38f14-4444-484f-832e-678344746d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:23.369364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0d66bd68454ea196bf51de5c9c4f82)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:23.370447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38f14-4444-484f-832e-678344746d79 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:23.370799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38f14-4444-484f-832e-678344746d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:23.371141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38f14-4444-484f-832e-678344746d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:23.372975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:23.379140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fbc38f14-4444-484f-832e-678344746d79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:23.406801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc38f14-4444-484f-832e-678344746d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:23.408264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc38f14-4444-484f-832e-678344746d79 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:23.440661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 894/1794] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:09:23.618897 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0d66bd68454ea196bf51de5c9c4f82'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:23.629123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0d66bd68454ea196bf51de5c9c4f82'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:23.633098 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0d66bd68454ea196bf51de5c9c4f82'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:23.635058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0d66bd68454ea196bf51de5c9c4f82'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:23.663387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a31e8a5-b6be-462e-a51b-2a5851473cbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0d66bd68454ea196bf51de5c9c4f82'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:23.766827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 901/1795] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:23 2022] DELETE /identity/v3/users/bd0d66bd68454ea196bf51de5c9c4f82 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:24.700228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc38f14-4444-484f-832e-678344746d79 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:24.710441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc38f14-4444-484f-832e-678344746d79 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:24.775094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:24.776079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:24.776430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:24.776794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] PATH_INFO: `/v3/projects/1d4ba2819fc84445a2592208b034b9fb` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:24.778064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4ba2819fc84445a2592208b034b9fb)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:24.780036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:24.921551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4ba2819fc84445a2592208b034b9fb', 'request_id': 'req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:24.932178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4ba2819fc84445a2592208b034b9fb', 'request_id': 'req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:24.951299 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:25.040412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4ba2819fc84445a2592208b034b9fb', 'request_id': 'req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:25.068487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] The token cache is being invalidate because project 1d4ba2819fc84445a2592208b034b9fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:25.069245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:25.071116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7cdc4bbb-c68b-4327-8aa8-cd44a0e9604c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:25.073810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 895/1796] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:24 2022] DELETE /identity/v3/projects/1d4ba2819fc84445a2592208b034b9fb => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:25.078518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89320ed6-e488-4345-80a2-e5375f7908f0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:25.079092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89320ed6-e488-4345-80a2-e5375f7908f0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:25.079092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89320ed6-e488-4345-80a2-e5375f7908f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:25.112502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-89320ed6-e488-4345-80a2-e5375f7908f0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:25.113812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-89320ed6-e488-4345-80a2-e5375f7908f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:25.181889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:25.266832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 902/1797] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:25.273599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae75e7-b6c7-4545-a473-325b3069bf81 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:25.273918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae75e7-b6c7-4545-a473-325b3069bf81 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:25.275883 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae75e7-b6c7-4545-a473-325b3069bf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:25.301717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d1ae75e7-b6c7-4545-a473-325b3069bf81 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:25.308836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d1ae75e7-b6c7-4545-a473-325b3069bf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:25.320646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 896/1798] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:25.324949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89320ed6-e488-4345-80a2-e5375f7908f0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:25.338486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89320ed6-e488-4345-80a2-e5375f7908f0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:25.358395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c3dec-5e03-4e75-92fa-c747a6e70b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:25.360605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c3dec-5e03-4e75-92fa-c747a6e70b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:25.361308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c3dec-5e03-4e75-92fa-c747a6e70b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:25.362267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c3dec-5e03-4e75-92fa-c747a6e70b42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:25.363084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c3dec-5e03-4e75-92fa-c747a6e70b42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:09:25.363552 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c3dec-5e03-4e75-92fa-c747a6e70b42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:25.370200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c3dec-5e03-4e75-92fa-c747a6e70b42 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:25.406299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 903/1799] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:09:25.407847 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae75e7-b6c7-4545-a473-325b3069bf81 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:25.428165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae75e7-b6c7-4545-a473-325b3069bf81 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:25.460130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d48600-e75a-4207-b2e5-610f8af89b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:25.461128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d48600-e75a-4207-b2e5-610f8af89b10 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:25.466035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d48600-e75a-4207-b2e5-610f8af89b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:25.466502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d48600-e75a-4207-b2e5-610f8af89b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:25.467376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d48600-e75a-4207-b2e5-610f8af89b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:25.471349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d48600-e75a-4207-b2e5-610f8af89b10 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:25.606896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 897/1800] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:25 2022] POST /identity/v3/projects => generated 371 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:25.611505 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c3dec-5e03-4e75-92fa-c747a6e70b42 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:25.625228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c3dec-5e03-4e75-92fa-c747a6e70b42 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:25.694846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9748a-6879-4a99-91c7-8c11cdf79414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:25.695693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9748a-6879-4a99-91c7-8c11cdf79414 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:25.695894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9748a-6879-4a99-91c7-8c11cdf79414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:25.696609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9748a-6879-4a99-91c7-8c11cdf79414 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:25.697346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9748a-6879-4a99-91c7-8c11cdf79414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:25.699330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9748a-6879-4a99-91c7-8c11cdf79414 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:25.828201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 904/1801] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:25 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:25.832329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d48600-e75a-4207-b2e5-610f8af89b10 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:25.850697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d48600-e75a-4207-b2e5-610f8af89b10 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:25.854771 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475dcde-69d3-4501-9bbf-0bf9af469fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:25.855532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475dcde-69d3-4501-9bbf-0bf9af469fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:25.856028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475dcde-69d3-4501-9bbf-0bf9af469fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:25.856381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475dcde-69d3-4501-9bbf-0bf9af469fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:25.857348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475dcde-69d3-4501-9bbf-0bf9af469fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:25.859303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475dcde-69d3-4501-9bbf-0bf9af469fe7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:25.873031 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 898/1802] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:25.875448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9748a-6879-4a99-91c7-8c11cdf79414 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:25.886642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9748a-6879-4a99-91c7-8c11cdf79414 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:25.891328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d949c-b5b1-472f-9d55-386e0230f1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:25.892022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d949c-b5b1-472f-9d55-386e0230f1da admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:25.892307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d949c-b5b1-472f-9d55-386e0230f1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:25.892584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d949c-b5b1-472f-9d55-386e0230f1da admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:25.893379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d949c-b5b1-472f-9d55-386e0230f1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:25.895236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d949c-b5b1-472f-9d55-386e0230f1da admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:25.909347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 905/1803] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:25.913627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475dcde-69d3-4501-9bbf-0bf9af469fe7 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:25.929386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475dcde-69d3-4501-9bbf-0bf9af469fe7 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:25.950587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c839d2-2908-4aaa-af7f-8b20fa91c246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:25.950587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c839d2-2908-4aaa-af7f-8b20fa91c246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:25.950587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c839d2-2908-4aaa-af7f-8b20fa91c246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:25.950587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c839d2-2908-4aaa-af7f-8b20fa91c246 admin admin] PATH_INFO: `/v3/projects/fab222ce3b0041329c6971676dedd756/users/03f47d055a894b9bb8b423ddfc3efeb0/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:25.969307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c839d2-2908-4aaa-af7f-8b20fa91c246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab222ce3b0041329c6971676dedd756, user_id=03f47d055a894b9bb8b423ddfc3efeb0, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:25.972050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c839d2-2908-4aaa-af7f-8b20fa91c246 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:26.078884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 899/1804] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:25 2022] PUT /identity/v3/projects/fab222ce3b0041329c6971676dedd756/users/03f47d055a894b9bb8b423ddfc3efeb0/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:26.084890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd18c6e-b318-4e1b-8613-201db2f8dbca None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:26.085956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd18c6e-b318-4e1b-8613-201db2f8dbca None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:26.086347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd18c6e-b318-4e1b-8613-201db2f8dbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:26.091396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e282c-b70e-44ac-96e0-4b4e7d2e9bb3 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:26.093103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e282c-b70e-44ac-96e0-4b4e7d2e9bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:26.093531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e282c-b70e-44ac-96e0-4b4e7d2e9bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:26.126883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1dd18c6e-b318-4e1b-8613-201db2f8dbca None None] MFA Rules not processed for user `66375274e9fb44d39936ebf64af69152`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:26.128851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1dd18c6e-b318-4e1b-8613-201db2f8dbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:26.148958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-155e282c-b70e-44ac-96e0-4b4e7d2e9bb3 None None] MFA Rules not processed for user `03f47d055a894b9bb8b423ddfc3efeb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:26.150749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-155e282c-b70e-44ac-96e0-4b4e7d2e9bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:26.227404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 906/1805] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:26.231721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd18c6e-b318-4e1b-8613-201db2f8dbca None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:26.233232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1dd18c6e-b318-4e1b-8613-201db2f8dbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:26.251481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 900/1806] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:26.256746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e282c-b70e-44ac-96e0-4b4e7d2e9bb3 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:26.274870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:26.346894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd18c6e-b318-4e1b-8613-201db2f8dbca None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:26.354194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4382a-9fe0-4af3-8c8f-a73d6cec0e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:26.355190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4382a-9fe0-4af3-8c8f-a73d6cec0e12 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:26.355888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4382a-9fe0-4af3-8c8f-a73d6cec0e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:26.356329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4382a-9fe0-4af3-8c8f-a73d6cec0e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:26.357995 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4382a-9fe0-4af3-8c8f-a73d6cec0e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:26.360538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4382a-9fe0-4af3-8c8f-a73d6cec0e12 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:26.380854 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e282c-b70e-44ac-96e0-4b4e7d2e9bb3 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:26.385878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cd671-e6f6-4d87-bb87-a939a77dabce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:26.387187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cd671-e6f6-4d87-bb87-a939a77dabce admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:26.387531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cd671-e6f6-4d87-bb87-a939a77dabce admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:26.387892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cd671-e6f6-4d87-bb87-a939a77dabce admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:26.388604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4cd671-e6f6-4d87-bb87-a939a77dabce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:26.394681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4cd671-e6f6-4d87-bb87-a939a77dabce admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:26.527325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 907/1807] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:26 2022] POST /identity/v3/projects => generated 373 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:09:26.529667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4382a-9fe0-4af3-8c8f-a73d6cec0e12 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:26.543858 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:26.569798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 901/1808] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:26 2022] POST /identity/v3/projects => generated 371 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:26.573808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cd671-e6f6-4d87-bb87-a939a77dabce admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:26.602529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4382a-9fe0-4af3-8c8f-a73d6cec0e12 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:26.640261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cd671-e6f6-4d87-bb87-a939a77dabce admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:26.645173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e1e91-b97f-43d5-9fe5-c8f49ef06d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:26.646091 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e1e91-b97f-43d5-9fe5-c8f49ef06d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:26.647146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e1e91-b97f-43d5-9fe5-c8f49ef06d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:26.647490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e1e91-b97f-43d5-9fe5-c8f49ef06d5f admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:26.648389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e1e91-b97f-43d5-9fe5-c8f49ef06d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:26.650900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e1e91-b97f-43d5-9fe5-c8f49ef06d5f admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:26.664537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120d1ec-0c83-4256-8092-615fbb5cb84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:26.665228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120d1ec-0c83-4256-8092-615fbb5cb84c admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:26.665508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120d1ec-0c83-4256-8092-615fbb5cb84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:26.665783 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120d1ec-0c83-4256-8092-615fbb5cb84c admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:26.666469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120d1ec-0c83-4256-8092-615fbb5cb84c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:26.668438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120d1ec-0c83-4256-8092-615fbb5cb84c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:27.001405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 908/1809] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:26 2022] POST /identity/v3/users => generated 293 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:27.002000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 902/1810] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:26 2022] POST /identity/v3/users => generated 292 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:27.006087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120d1ec-0c83-4256-8092-615fbb5cb84c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:27.006240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e1e91-b97f-43d5-9fe5-c8f49ef06d5f admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:27.016531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e1e91-b97f-43d5-9fe5-c8f49ef06d5f admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:27.016753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120d1ec-0c83-4256-8092-615fbb5cb84c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:27.021374 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b094d06-9ed9-4ccf-abfb-e2317f7201bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:27.021634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ef493-7e50-469d-a1b3-a99a594d41e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:27.022047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094d06-9ed9-4ccf-abfb-e2317f7201bc admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.022159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ef493-7e50-469d-a1b3-a99a594d41e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.022320 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094d06-9ed9-4ccf-abfb-e2317f7201bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.022444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ef493-7e50-469d-a1b3-a99a594d41e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.022619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ef493-7e50-469d-a1b3-a99a594d41e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:27.022715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094d06-9ed9-4ccf-abfb-e2317f7201bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:27.023349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ef493-7e50-469d-a1b3-a99a594d41e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:27.023455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b094d06-9ed9-4ccf-abfb-e2317f7201bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:27.026191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ef493-7e50-469d-a1b3-a99a594d41e3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:27.026317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b094d06-9ed9-4ccf-abfb-e2317f7201bc admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:27.035775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 909/1811] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:27.038156 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 903/1812] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:27.042389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b094d06-9ed9-4ccf-abfb-e2317f7201bc admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:27.042536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ef493-7e50-469d-a1b3-a99a594d41e3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:27.053306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b094d06-9ed9-4ccf-abfb-e2317f7201bc admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:27.054452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ef493-7e50-469d-a1b3-a99a594d41e3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:27.057907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb19ace-c44b-4328-b30c-e397d8a48331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:27.058781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb19ace-c44b-4328-b30c-e397d8a48331 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.059064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb19ace-c44b-4328-b30c-e397d8a48331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.059177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad4348-ffb0-48e9-a4da-09dc245cb92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:27.059383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb19ace-c44b-4328-b30c-e397d8a48331 admin admin] PATH_INFO: `/v3/projects/efc4ae54a9f748bb971a3488b560d7ae/users/e1446cf4141e4fe19a5bdb3c65f2dee7/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:27.060064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad4348-ffb0-48e9-a4da-09dc245cb92d admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.060355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad4348-ffb0-48e9-a4da-09dc245cb92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.060631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad4348-ffb0-48e9-a4da-09dc245cb92d admin admin] PATH_INFO: `/v3/projects/f8f4429bcd344015956046f2802063de/users/e6ec9086552b4b26a34c892ded10f739/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:27.077469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb19ace-c44b-4328-b30c-e397d8a48331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc4ae54a9f748bb971a3488b560d7ae, user_id=e1446cf4141e4fe19a5bdb3c65f2dee7, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:27.080241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad4348-ffb0-48e9-a4da-09dc245cb92d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8f4429bcd344015956046f2802063de, user_id=e6ec9086552b4b26a34c892ded10f739, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:27.080884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb19ace-c44b-4328-b30c-e397d8a48331 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:27.081982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad4348-ffb0-48e9-a4da-09dc245cb92d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:27.198735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 910/1813] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:27 2022] PUT /identity/v3/projects/f8f4429bcd344015956046f2802063de/users/e6ec9086552b4b26a34c892ded10f739/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:27.199892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 904/1814] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:27 2022] PUT /identity/v3/projects/efc4ae54a9f748bb971a3488b560d7ae/users/e1446cf4141e4fe19a5bdb3c65f2dee7/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:27.202630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad4348-ffb0-48e9-a4da-09dc245cb92d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:27.203449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb19ace-c44b-4328-b30c-e397d8a48331 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:27.267219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad4348-ffb0-48e9-a4da-09dc245cb92d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:27.267426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb19ace-c44b-4328-b30c-e397d8a48331 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:27.271624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795daf-74ac-45e1-a0a7-82f83751110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:27.272292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795daf-74ac-45e1-a0a7-82f83751110c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.272512 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7aef2-71fb-4658-b751-9534f9bf1a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:27.272638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795daf-74ac-45e1-a0a7-82f83751110c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.272880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795daf-74ac-45e1-a0a7-82f83751110c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:27.277738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7aef2-71fb-4658-b751-9534f9bf1a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.277738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7aef2-71fb-4658-b751-9534f9bf1a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.277943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4795daf-74ac-45e1-a0a7-82f83751110c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:27.278017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7aef2-71fb-4658-b751-9534f9bf1a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:27.278017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7aef2-71fb-4658-b751-9534f9bf1a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:27.278150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4795daf-74ac-45e1-a0a7-82f83751110c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:27.278229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7aef2-71fb-4658-b751-9534f9bf1a4e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:27.285905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 911/1815] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:27.292923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 905/1816] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:09:27.292923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7aef2-71fb-4658-b751-9534f9bf1a4e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:27.293175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795daf-74ac-45e1-a0a7-82f83751110c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:27.304466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795daf-74ac-45e1-a0a7-82f83751110c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:27.308979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7aef2-71fb-4658-b751-9534f9bf1a4e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:27.309155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63617363-33a3-4142-958f-33e80bde4484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:27.310132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63617363-33a3-4142-958f-33e80bde4484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.310417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63617363-33a3-4142-958f-33e80bde4484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.310703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63617363-33a3-4142-958f-33e80bde4484 admin admin] PATH_INFO: `/v3/projects/efc4ae54a9f748bb971a3488b560d7ae/users/e1446cf4141e4fe19a5bdb3c65f2dee7/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:27.312653 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63617363-33a3-4142-958f-33e80bde4484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc4ae54a9f748bb971a3488b560d7ae, user_id=e1446cf4141e4fe19a5bdb3c65f2dee7, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:27.312782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103be970-c1fb-4ac0-bad0-5c0cb9022798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:27.313645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103be970-c1fb-4ac0-bad0-5c0cb9022798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.313869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103be970-c1fb-4ac0-bad0-5c0cb9022798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.314223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103be970-c1fb-4ac0-bad0-5c0cb9022798 admin admin] PATH_INFO: `/v3/projects/f8f4429bcd344015956046f2802063de/users/e6ec9086552b4b26a34c892ded10f739/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:27.315377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63617363-33a3-4142-958f-33e80bde4484 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:27.316169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103be970-c1fb-4ac0-bad0-5c0cb9022798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8f4429bcd344015956046f2802063de, user_id=e6ec9086552b4b26a34c892ded10f739, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:27.318858 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103be970-c1fb-4ac0-bad0-5c0cb9022798 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:27.327144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 912/1817] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:27 2022] PUT /identity/v3/projects/efc4ae54a9f748bb971a3488b560d7ae/users/e1446cf4141e4fe19a5bdb3c65f2dee7/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:09:27.331229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 906/1818] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:27 2022] PUT /identity/v3/projects/f8f4429bcd344015956046f2802063de/users/e6ec9086552b4b26a34c892ded10f739/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:27.348186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6233da8-94bf-4e42-93e8-7bfadf7b2d31 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.348186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6233da8-94bf-4e42-93e8-7bfadf7b2d31 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.348186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6233da8-94bf-4e42-93e8-7bfadf7b2d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:27.372642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b6233da8-94bf-4e42-93e8-7bfadf7b2d31 None None] MFA Rules not processed for user `e1446cf4141e4fe19a5bdb3c65f2dee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:27.373933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b6233da8-94bf-4e42-93e8-7bfadf7b2d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:27.468506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:27.480791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 913/1819] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:27.495984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103be970-c1fb-4ac0-bad0-5c0cb9022798 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:27.501546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-103be970-c1fb-4ac0-bad0-5c0cb9022798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:27.658538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103be970-c1fb-4ac0-bad0-5c0cb9022798 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:27.663965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31989b0-069d-475d-90e1-3d08bb65fa81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:27.664748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31989b0-069d-475d-90e1-3d08bb65fa81 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.665133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31989b0-069d-475d-90e1-3d08bb65fa81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.665770 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31989b0-069d-475d-90e1-3d08bb65fa81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:27.679261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31989b0-069d-475d-90e1-3d08bb65fa81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:27.680914 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31989b0-069d-475d-90e1-3d08bb65fa81 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:27.692294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 907/1820] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:27.944869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6233da8-94bf-4e42-93e8-7bfadf7b2d31 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:27.964551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6233da8-94bf-4e42-93e8-7bfadf7b2d31 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:27.968848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f21b5-6acc-451b-8498-bbfb1938e403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:27.969725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f21b5-6acc-451b-8498-bbfb1938e403 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:27.970150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f21b5-6acc-451b-8498-bbfb1938e403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:27.970587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f21b5-6acc-451b-8498-bbfb1938e403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:28.077731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f21b5-6acc-451b-8498-bbfb1938e403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:28.080161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f21b5-6acc-451b-8498-bbfb1938e403 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:28.091881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 914/1821] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:28.269644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:31.124333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f86850-753f-420b-adf1-46b702ca489e None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:31.131437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f86850-753f-420b-adf1-46b702ca489e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:31.131437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f86850-753f-420b-adf1-46b702ca489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:31.157089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-07f86850-753f-420b-adf1-46b702ca489e None None] MFA Rules not processed for user `e6ec9086552b4b26a34c892ded10f739`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:31.158348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-07f86850-753f-420b-adf1-46b702ca489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:31.263556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 908/1822] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:31.272064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f21b5-6acc-451b-8498-bbfb1938e403 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:31.284574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f21b5-6acc-451b-8498-bbfb1938e403 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:31.291381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a056ae5-a092-4d63-a363-e8beaf0a38b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:31.296715 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a056ae5-a092-4d63-a363-e8beaf0a38b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:31.297276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a056ae5-a092-4d63-a363-e8beaf0a38b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:31.298098 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a056ae5-a092-4d63-a363-e8beaf0a38b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:31.320883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:31.378132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a056ae5-a092-4d63-a363-e8beaf0a38b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:31.381042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a056ae5-a092-4d63-a363-e8beaf0a38b7 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:31.395839 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 915/1823] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:33.236478 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5a6fb-047c-4934-bc3f-281402865974 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:33.237420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5a6fb-047c-4934-bc3f-281402865974 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:33.237845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5a6fb-047c-4934-bc3f-281402865974 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:33.240361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 909/1824] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:09:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:09:33.261464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa38b3-6955-4764-9a6d-3f76966439c8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:33.262452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa38b3-6955-4764-9a6d-3f76966439c8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:33.262887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa38b3-6955-4764-9a6d-3f76966439c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:33.292011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4aaa38b3-6955-4764-9a6d-3f76966439c8 None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:33.296101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4aaa38b3-6955-4764-9a6d-3f76966439c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:33.382526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 916/1825] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:09:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:33.438475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5a6fb-047c-4934-bc3f-281402865974 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:33.440523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1ce5a6fb-047c-4934-bc3f-281402865974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:33.531554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5a6fb-047c-4934-bc3f-281402865974 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:33.540413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:33.540413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:33.540413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:33.540413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] PATH_INFO: `/v3/users/a11c71934a5b488f9b933c307d8d4f03` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:33.542708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11c71934a5b488f9b933c307d8d4f03)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:33.545335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:33.561357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa38b3-6955-4764-9a6d-3f76966439c8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:33.563401 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4aaa38b3-6955-4764-9a6d-3f76966439c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:33.580225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa38b3-6955-4764-9a6d-3f76966439c8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:33.586456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b4780-2386-464f-8216-9b83612652df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:33.591504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b4780-2386-464f-8216-9b83612652df service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:33.591981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b4780-2386-464f-8216-9b83612652df service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:33.592977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b4780-2386-464f-8216-9b83612652df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:33.611983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b4780-2386-464f-8216-9b83612652df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:33.614446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b4780-2386-464f-8216-9b83612652df service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:33.632958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 917/1826] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:33.813486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11c71934a5b488f9b933c307d8d4f03'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:33.821259 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11c71934a5b488f9b933c307d8d4f03'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:33.821990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11c71934a5b488f9b933c307d8d4f03'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:33.830096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:33.835920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11c71934a5b488f9b933c307d8d4f03'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:33.836662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11c71934a5b488f9b933c307d8d4f03'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:33.919626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 910/1827] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:33 2022] DELETE /identity/v3/users/a11c71934a5b488f9b933c307d8d4f03 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:33.924589 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d6604-e7fe-4f25-9f49-ed51d798a2d5 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:33.924972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d6604-e7fe-4f25-9f49-ed51d798a2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:33.925261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d6604-e7fe-4f25-9f49-ed51d798a2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:33.944643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-618d6604-e7fe-4f25-9f49-ed51d798a2d5 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:33.945908 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-618d6604-e7fe-4f25-9f49-ed51d798a2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:34.012092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 918/1828] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:34.025842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:34.027367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:34.085312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c033f7d-18e5-486c-8136-cd4e04f4fcd6 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:34.089028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c01468-c645-4739-905b-b40799016818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:34.089699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c01468-c645-4739-905b-b40799016818 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:34.090063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c01468-c645-4739-905b-b40799016818 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:34.090341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c01468-c645-4739-905b-b40799016818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:34.101133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c01468-c645-4739-905b-b40799016818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:34.102806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c01468-c645-4739-905b-b40799016818 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:34.112955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 911/1829] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:34.507795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d6604-e7fe-4f25-9f49-ed51d798a2d5 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:34.509359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-618d6604-e7fe-4f25-9f49-ed51d798a2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:34.525073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d6604-e7fe-4f25-9f49-ed51d798a2d5 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:34.530532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:34.531393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:34.531749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:34.536440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] PATH_INFO: `/v3/projects/8195cf24d54d49868ea9786bcdb1ff84` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:34.536440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8195cf24d54d49868ea9786bcdb1ff84)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:34.536440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:34.659979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:34.679450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8195cf24d54d49868ea9786bcdb1ff84', 'request_id': 'req-35ff455f-279e-4d4b-90ba-89ab2ca62076'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:34.684996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8195cf24d54d49868ea9786bcdb1ff84', 'request_id': 'req-35ff455f-279e-4d4b-90ba-89ab2ca62076'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:34.697283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8195cf24d54d49868ea9786bcdb1ff84', 'request_id': 'req-35ff455f-279e-4d4b-90ba-89ab2ca62076'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:34.785197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] The token cache is being invalidate because project 8195cf24d54d49868ea9786bcdb1ff84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:34.785493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:34.786174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:34.789016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 919/1830] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:34 2022] DELETE /identity/v3/projects/8195cf24d54d49868ea9786bcdb1ff84 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:34.792552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c01468-c645-4739-905b-b40799016818 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:34.803090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c01468-c645-4739-905b-b40799016818 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:34.857430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:34.858287 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:34.858571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:34.858850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] PATH_INFO: `/v3/users/d1149191ade74b5eb56d10a94c7d71f6` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:34.860303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1149191ade74b5eb56d10a94c7d71f6)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:34.862367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:35.077893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1149191ade74b5eb56d10a94c7d71f6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:35.087909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1149191ade74b5eb56d10a94c7d71f6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:35.088723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1149191ade74b5eb56d10a94c7d71f6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:35.101922 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1149191ade74b5eb56d10a94c7d71f6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:35.102709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5d8e0a3f-e9ef-4279-a273-6b50413dffcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1149191ade74b5eb56d10a94c7d71f6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:35.186860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 912/1831] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:34 2022] DELETE /identity/v3/users/d1149191ade74b5eb56d10a94c7d71f6 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:35.647002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:35.648965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:35.662086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff455f-279e-4d4b-90ba-89ab2ca62076 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:35.719931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:35.720809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:35.721093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:35.721368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] PATH_INFO: `/v3/projects/6b2d4d432c54453793ba02df411012e5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:35.722538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b2d4d432c54453793ba02df411012e5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:35.724466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:35.875239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b2d4d432c54453793ba02df411012e5', 'request_id': 'req-5c4b694a-972b-4b07-9ad8-96d818d171eb'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:35.881527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b2d4d432c54453793ba02df411012e5', 'request_id': 'req-5c4b694a-972b-4b07-9ad8-96d818d171eb'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:35.888248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b2d4d432c54453793ba02df411012e5', 'request_id': 'req-5c4b694a-972b-4b07-9ad8-96d818d171eb'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:35.987986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] The token cache is being invalidate because project 6b2d4d432c54453793ba02df411012e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:35.988272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:35.988920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5c4b694a-972b-4b07-9ad8-96d818d171eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:35.990526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 920/1832] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:35 2022] DELETE /identity/v3/projects/6b2d4d432c54453793ba02df411012e5 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:35.995473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68163781-c4bc-4f1a-9e86-e1c18d719eba None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:35.995970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68163781-c4bc-4f1a-9e86-e1c18d719eba None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:35.996057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68163781-c4bc-4f1a-9e86-e1c18d719eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:36.004191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:36.014196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-68163781-c4bc-4f1a-9e86-e1c18d719eba None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:36.015336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-68163781-c4bc-4f1a-9e86-e1c18d719eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:36.075530 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 913/1833] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:36.083026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b69b12-3082-4caa-87cd-da28e09e75f2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:36.083931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b69b12-3082-4caa-87cd-da28e09e75f2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:36.084265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b69b12-3082-4caa-87cd-da28e09e75f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:36.102583 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-72b69b12-3082-4caa-87cd-da28e09e75f2 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:36.103787 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-72b69b12-3082-4caa-87cd-da28e09e75f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:36.111942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 921/1834] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:36.116072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68163781-c4bc-4f1a-9e86-e1c18d719eba None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:36.125486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68163781-c4bc-4f1a-9e86-e1c18d719eba None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:36.130509 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e33fa-dcb6-4477-bec4-4593957f86d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:36.131149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e33fa-dcb6-4477-bec4-4593957f86d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:36.131429 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e33fa-dcb6-4477-bec4-4593957f86d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:36.131703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e33fa-dcb6-4477-bec4-4593957f86d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:36.132419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e33fa-dcb6-4477-bec4-4593957f86d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:09:36.132721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e33fa-dcb6-4477-bec4-4593957f86d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:36.134323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e33fa-dcb6-4477-bec4-4593957f86d1 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:36.146742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 914/1835] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:09:36.150826 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b69b12-3082-4caa-87cd-da28e09e75f2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:36.159970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b69b12-3082-4caa-87cd-da28e09e75f2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:36.164607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6a6ea-71bf-452a-bc4f-4da907f25651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:36.165307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6a6ea-71bf-452a-bc4f-4da907f25651 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:36.165597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6a6ea-71bf-452a-bc4f-4da907f25651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:36.165872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6a6ea-71bf-452a-bc4f-4da907f25651 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:36.166595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6a6ea-71bf-452a-bc4f-4da907f25651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:36.168576 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6a6ea-71bf-452a-bc4f-4da907f25651 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:36.291199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 922/1836] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:36 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:36.292448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e33fa-dcb6-4477-bec4-4593957f86d1 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:36.302758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e33fa-dcb6-4477-bec4-4593957f86d1 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:36.365853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9149341-a28c-4299-825c-9799eb2501b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:36.366536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9149341-a28c-4299-825c-9799eb2501b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:36.366821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9149341-a28c-4299-825c-9799eb2501b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:36.367100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9149341-a28c-4299-825c-9799eb2501b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:36.367851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9149341-a28c-4299-825c-9799eb2501b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:36.369791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9149341-a28c-4299-825c-9799eb2501b1 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:36.461506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 915/1837] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:36 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:36.466899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6a6ea-71bf-452a-bc4f-4da907f25651 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:36.477389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6a6ea-71bf-452a-bc4f-4da907f25651 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:36.482045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c0be0-854d-4189-8aef-fc4d863b6b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:36.482710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c0be0-854d-4189-8aef-fc4d863b6b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:36.482997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c0be0-854d-4189-8aef-fc4d863b6b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:36.483275 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c0be0-854d-4189-8aef-fc4d863b6b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:36.484287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c0be0-854d-4189-8aef-fc4d863b6b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:36.485914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c0be0-854d-4189-8aef-fc4d863b6b8c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:36.497278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 923/1838] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:36.501368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9149341-a28c-4299-825c-9799eb2501b1 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:36.512176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9149341-a28c-4299-825c-9799eb2501b1 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:36.516970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584ca47-eb85-4c41-a197-028fee714b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:36.517643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584ca47-eb85-4c41-a197-028fee714b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:36.517931 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584ca47-eb85-4c41-a197-028fee714b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:36.518207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584ca47-eb85-4c41-a197-028fee714b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:36.518934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584ca47-eb85-4c41-a197-028fee714b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:36.520593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584ca47-eb85-4c41-a197-028fee714b13 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:36.533072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 916/1839] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:09:36.537621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c0be0-854d-4189-8aef-fc4d863b6b8c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:36.544537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:36.549990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c0be0-854d-4189-8aef-fc4d863b6b8c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:36.557311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819df0b2-d542-4c83-8b95-d266a1945e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:36.558295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819df0b2-d542-4c83-8b95-d266a1945e1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:36.558624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819df0b2-d542-4c83-8b95-d266a1945e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:36.559038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819df0b2-d542-4c83-8b95-d266a1945e1b admin admin] PATH_INFO: `/v3/projects/c1e18c502f8a49cda53ffd192d8d4485/users/2cc651597b92483b9d5f39a430e805e6/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:36.577931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819df0b2-d542-4c83-8b95-d266a1945e1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e18c502f8a49cda53ffd192d8d4485, user_id=2cc651597b92483b9d5f39a430e805e6, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:36.581106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819df0b2-d542-4c83-8b95-d266a1945e1b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:36.670252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 924/1840] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:36 2022] PUT /identity/v3/projects/c1e18c502f8a49cda53ffd192d8d4485/users/2cc651597b92483b9d5f39a430e805e6/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:36.674741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a930d8-43bc-4fbd-877a-7b7862e29699 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:36.675036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a930d8-43bc-4fbd-877a-7b7862e29699 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:36.675329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a930d8-43bc-4fbd-877a-7b7862e29699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:36.704947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-03a930d8-43bc-4fbd-877a-7b7862e29699 None None] MFA Rules not processed for user `2cc651597b92483b9d5f39a430e805e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:36.706244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-03a930d8-43bc-4fbd-877a-7b7862e29699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:36.779755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:36.783630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 917/1841] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:36.788218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819df0b2-d542-4c83-8b95-d266a1945e1b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:36.869573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819df0b2-d542-4c83-8b95-d266a1945e1b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:36.878004 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803499bd-0661-4f2e-8ad1-a70c68b38915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:36.879002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803499bd-0661-4f2e-8ad1-a70c68b38915 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:36.879416 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803499bd-0661-4f2e-8ad1-a70c68b38915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:36.879984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803499bd-0661-4f2e-8ad1-a70c68b38915 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:36.881117 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803499bd-0661-4f2e-8ad1-a70c68b38915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:36.884174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803499bd-0661-4f2e-8ad1-a70c68b38915 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:36.994466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 925/1842] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:36 2022] POST /identity/v3/projects => generated 377 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:36.996099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a930d8-43bc-4fbd-877a-7b7862e29699 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:37.008500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a930d8-43bc-4fbd-877a-7b7862e29699 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:37.091805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7db1f-b414-410e-a6c4-f5f7a3259690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:37.093098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7db1f-b414-410e-a6c4-f5f7a3259690 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:37.093383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7db1f-b414-410e-a6c4-f5f7a3259690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:37.093659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7db1f-b414-410e-a6c4-f5f7a3259690 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:37.094365 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7db1f-b414-410e-a6c4-f5f7a3259690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:37.096369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7db1f-b414-410e-a6c4-f5f7a3259690 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:37.167666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 918/1843] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:36 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:37.172323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803499bd-0661-4f2e-8ad1-a70c68b38915 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:37.190760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803499bd-0661-4f2e-8ad1-a70c68b38915 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:37.196495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc51b1-a057-451e-97d5-c49cb9b69137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:37.197328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc51b1-a057-451e-97d5-c49cb9b69137 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:37.197612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc51b1-a057-451e-97d5-c49cb9b69137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:37.197884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc51b1-a057-451e-97d5-c49cb9b69137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:37.199578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc51b1-a057-451e-97d5-c49cb9b69137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:37.200802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc51b1-a057-451e-97d5-c49cb9b69137 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:37.220781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 926/1844] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:37.225756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7db1f-b414-410e-a6c4-f5f7a3259690 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:37.238289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7db1f-b414-410e-a6c4-f5f7a3259690 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:37.243261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b120ad-253b-46ef-a06e-b5ac929ea45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:37.243948 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b120ad-253b-46ef-a06e-b5ac929ea45a admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:37.244235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b120ad-253b-46ef-a06e-b5ac929ea45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:37.244520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b120ad-253b-46ef-a06e-b5ac929ea45a admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:37.245649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b120ad-253b-46ef-a06e-b5ac929ea45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:37.249047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b120ad-253b-46ef-a06e-b5ac929ea45a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:37.262587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 919/1845] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:37.267220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc51b1-a057-451e-97d5-c49cb9b69137 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:37.281816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc51b1-a057-451e-97d5-c49cb9b69137 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:37.296526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34a61-479e-4d7c-bba5-13a953598fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:37.297903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34a61-479e-4d7c-bba5-13a953598fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:37.298292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34a61-479e-4d7c-bba5-13a953598fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:37.298790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34a61-479e-4d7c-bba5-13a953598fcb admin admin] PATH_INFO: `/v3/projects/b96d9d2bad344232bc797b35a7440a89/users/1b7c9c577e2c4ed5b423a42d844ac585/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:37.324333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be34a61-479e-4d7c-bba5-13a953598fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96d9d2bad344232bc797b35a7440a89, user_id=1b7c9c577e2c4ed5b423a42d844ac585, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:37.327362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be34a61-479e-4d7c-bba5-13a953598fcb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:37.415934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 927/1846] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:37 2022] PUT /identity/v3/projects/b96d9d2bad344232bc797b35a7440a89/users/1b7c9c577e2c4ed5b423a42d844ac585/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:37.420564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42abf4-6912-465d-a664-0dec19dcb08e None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:37.420887 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42abf4-6912-465d-a664-0dec19dcb08e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:37.421171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42abf4-6912-465d-a664-0dec19dcb08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:37.450941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8a42abf4-6912-465d-a664-0dec19dcb08e None None] MFA Rules not processed for user `1b7c9c577e2c4ed5b423a42d844ac585`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:37.452214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8a42abf4-6912-465d-a664-0dec19dcb08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:37.546831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 920/1847] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:37.548105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34a61-479e-4d7c-bba5-13a953598fcb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:37.622798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34a61-479e-4d7c-bba5-13a953598fcb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:37.632616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188913f6-74c9-4821-b05d-15a7d8f36be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:37.633592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188913f6-74c9-4821-b05d-15a7d8f36be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:37.634068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188913f6-74c9-4821-b05d-15a7d8f36be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:37.634486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188913f6-74c9-4821-b05d-15a7d8f36be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:37.635795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188913f6-74c9-4821-b05d-15a7d8f36be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:37.638090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188913f6-74c9-4821-b05d-15a7d8f36be4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:37.738205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42abf4-6912-465d-a664-0dec19dcb08e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:37.739796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8a42abf4-6912-465d-a664-0dec19dcb08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:37.773706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 928/1848] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:37 2022] POST /identity/v3/projects => generated 379 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:37.774635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188913f6-74c9-4821-b05d-15a7d8f36be4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:37.801882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:37.869525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188913f6-74c9-4821-b05d-15a7d8f36be4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:37.878831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0edfb-3f29-4892-a50c-eba58f788616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:37.879643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0edfb-3f29-4892-a50c-eba58f788616 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:37.886303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0edfb-3f29-4892-a50c-eba58f788616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:37.886303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0edfb-3f29-4892-a50c-eba58f788616 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:37.886303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0edfb-3f29-4892-a50c-eba58f788616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:37.886303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0edfb-3f29-4892-a50c-eba58f788616 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:37.888334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42abf4-6912-465d-a664-0dec19dcb08e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:37.920579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:37.961689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5ed3d-455d-4ab2-89fc-75ee283e6c6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:37.963112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ed3d-455d-4ab2-89fc-75ee283e6c6d service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:37.963516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ed3d-455d-4ab2-89fc-75ee283e6c6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:37.963898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ed3d-455d-4ab2-89fc-75ee283e6c6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:37.965956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5ef5ed3d-455d-4ab2-89fc-75ee283e6c6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:37.982317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 929/1849] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:37 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:37.984205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0edfb-3f29-4892-a50c-eba58f788616 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:37.995856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0edfb-3f29-4892-a50c-eba58f788616 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:38.008288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541b6c-32d1-4049-9ddf-b16dbd7aa6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:38.009240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541b6c-32d1-4049-9ddf-b16dbd7aa6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:38.009593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541b6c-32d1-4049-9ddf-b16dbd7aa6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:38.009939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541b6c-32d1-4049-9ddf-b16dbd7aa6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:38.010754 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4541b6c-32d1-4049-9ddf-b16dbd7aa6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:38.012466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4541b6c-32d1-4049-9ddf-b16dbd7aa6a3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:38.024815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 930/1850] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:38.029996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541b6c-32d1-4049-9ddf-b16dbd7aa6a3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:38.046750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541b6c-32d1-4049-9ddf-b16dbd7aa6a3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:38.048933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c00d4d-1ba2-4fda-b091-8d51e5344983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:38.049770 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5ed3d-455d-4ab2-89fc-75ee283e6c6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:38.050551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c00d4d-1ba2-4fda-b091-8d51e5344983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:38.050551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c00d4d-1ba2-4fda-b091-8d51e5344983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:38.050551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c00d4d-1ba2-4fda-b091-8d51e5344983 admin admin] PATH_INFO: `/v3/projects/7196748c50c74591bb23a9f92b4e4e36/users/67497d833f154678b815c521427164fd/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:38.051688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5ed3d-455d-4ab2-89fc-75ee283e6c6d service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:38.070065 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c00d4d-1ba2-4fda-b091-8d51e5344983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7196748c50c74591bb23a9f92b4e4e36, user_id=67497d833f154678b815c521427164fd, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:38.073627 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c00d4d-1ba2-4fda-b091-8d51e5344983 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:38.080867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 921/1851] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:38.204816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 931/1852] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:38 2022] PUT /identity/v3/projects/7196748c50c74591bb23a9f92b4e4e36/users/67497d833f154678b815c521427164fd/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:38.209885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5ed3d-455d-4ab2-89fc-75ee283e6c6d service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:38.220799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5ed3d-455d-4ab2-89fc-75ee283e6c6d service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:38.299494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fe170-9983-48d3-bbcb-a04718a164ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:38.300172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe170-9983-48d3-bbcb-a04718a164ab admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:38.300456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe170-9983-48d3-bbcb-a04718a164ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:38.300733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe170-9983-48d3-bbcb-a04718a164ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:38.301493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fe170-9983-48d3-bbcb-a04718a164ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:38.303905 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fe170-9983-48d3-bbcb-a04718a164ab admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:38.314514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 922/1853] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:38.319404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c00d4d-1ba2-4fda-b091-8d51e5344983 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:38.329872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c00d4d-1ba2-4fda-b091-8d51e5344983 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:38.334541 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f3c6f-8b02-4945-90f0-fd16dde37ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:38.335451 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f3c6f-8b02-4945-90f0-fd16dde37ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:38.335737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f3c6f-8b02-4945-90f0-fd16dde37ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:38.336018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f3c6f-8b02-4945-90f0-fd16dde37ecb admin admin] PATH_INFO: `/v3/projects/7196748c50c74591bb23a9f92b4e4e36/users/67497d833f154678b815c521427164fd/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:38.338052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f3c6f-8b02-4945-90f0-fd16dde37ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7196748c50c74591bb23a9f92b4e4e36, user_id=67497d833f154678b815c521427164fd, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:38.340730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f3c6f-8b02-4945-90f0-fd16dde37ecb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:38.352195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 932/1854] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:38 2022] PUT /identity/v3/projects/7196748c50c74591bb23a9f92b4e4e36/users/67497d833f154678b815c521427164fd/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:38.362609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99556638-fec1-4529-8223-db47c329e7d3 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:38.362907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99556638-fec1-4529-8223-db47c329e7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:38.363188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99556638-fec1-4529-8223-db47c329e7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:38.391621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-99556638-fec1-4529-8223-db47c329e7d3 None None] MFA Rules not processed for user `67497d833f154678b815c521427164fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:38.392850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-99556638-fec1-4529-8223-db47c329e7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:38.453368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:38.462529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 923/1855] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:38.476190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f3c6f-8b02-4945-90f0-fd16dde37ecb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:38.534246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f3c6f-8b02-4945-90f0-fd16dde37ecb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:38.538906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b16f6-8f8b-41cd-b52b-d41662a22656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:38.540069 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b16f6-8f8b-41cd-b52b-d41662a22656 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:38.540069 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b16f6-8f8b-41cd-b52b-d41662a22656 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:38.540405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b16f6-8f8b-41cd-b52b-d41662a22656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:38.550828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b16f6-8f8b-41cd-b52b-d41662a22656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:38.552312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b16f6-8f8b-41cd-b52b-d41662a22656 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:38.564374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 933/1856] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:09:38.689030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99556638-fec1-4529-8223-db47c329e7d3 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:38.699201 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99556638-fec1-4529-8223-db47c329e7d3 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:38.702790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa31301-ff65-48e6-bea2-45489d3473d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:38.703466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa31301-ff65-48e6-bea2-45489d3473d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:38.703750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa31301-ff65-48e6-bea2-45489d3473d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:38.704028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa31301-ff65-48e6-bea2-45489d3473d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:38.775684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa31301-ff65-48e6-bea2-45489d3473d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:38.777798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa31301-ff65-48e6-bea2-45489d3473d9 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:38.789623 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 924/1857] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:09:39.225527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935973eb-46b0-413d-92eb-817f099fd726 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:39.227457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935973eb-46b0-413d-92eb-817f099fd726 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:39.227935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935973eb-46b0-413d-92eb-817f099fd726 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:39.230441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 934/1858] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:09:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:09:39.248614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03af75-d49c-4cbd-9ee0-6347c420dc68 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:39.249219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03af75-d49c-4cbd-9ee0-6347c420dc68 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:39.249219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03af75-d49c-4cbd-9ee0-6347c420dc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:39.276275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-ba03af75-d49c-4cbd-9ee0-6347c420dc68 None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:39.280326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ba03af75-d49c-4cbd-9ee0-6347c420dc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:39.379509 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 925/1859] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:09:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:39.403647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935973eb-46b0-413d-92eb-817f099fd726 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:39.405417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-935973eb-46b0-413d-92eb-817f099fd726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:39.427875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935973eb-46b0-413d-92eb-817f099fd726 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:39.434358 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfbafc-e65e-4fb8-9b3e-f065408ce8de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:39.435274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfbafc-e65e-4fb8-9b3e-f065408ce8de service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:39.435751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfbafc-e65e-4fb8-9b3e-f065408ce8de service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:39.436127 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfbafc-e65e-4fb8-9b3e-f065408ce8de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:39.447303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfbafc-e65e-4fb8-9b3e-f065408ce8de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:39.449544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfbafc-e65e-4fb8-9b3e-f065408ce8de service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:39.463677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 935/1860] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:43.284681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03af75-d49c-4cbd-9ee0-6347c420dc68 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:43.374613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03af75-d49c-4cbd-9ee0-6347c420dc68 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:43.379933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:43.380854 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:43.381296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:43.384703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] PATH_INFO: `/v3/users/2cc651597b92483b9d5f39a430e805e6` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:43.384703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc651597b92483b9d5f39a430e805e6)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:43.384925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:43.490819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:43.651746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc651597b92483b9d5f39a430e805e6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:43.665741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc651597b92483b9d5f39a430e805e6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:43.667131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc651597b92483b9d5f39a430e805e6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:43.690489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc651597b92483b9d5f39a430e805e6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:43.691303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc651597b92483b9d5f39a430e805e6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:43.842324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 926/1861] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:43 2022] DELETE /identity/v3/users/2cc651597b92483b9d5f39a430e805e6 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:09:43.846994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3811e-ef1e-41b8-919c-5854e63a9b63 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:43.847395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3811e-ef1e-41b8-919c-5854e63a9b63 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:43.847492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3811e-ef1e-41b8-919c-5854e63a9b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:43.878153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-54b3811e-ef1e-41b8-919c-5854e63a9b63 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:43.878153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-54b3811e-ef1e-41b8-919c-5854e63a9b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:43.938786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:43.984529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 936/1862] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:09:43.984611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:43.984611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:44.123717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3371788-72dd-4705-b9b0-236899bf870b admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:44.127504 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5b42a-83a6-4995-8809-2724055325a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:44.128687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5b42a-83a6-4995-8809-2724055325a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:44.128687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5b42a-83a6-4995-8809-2724055325a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:44.128687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5b42a-83a6-4995-8809-2724055325a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:44.144002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5b42a-83a6-4995-8809-2724055325a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:44.146593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5b42a-83a6-4995-8809-2724055325a4 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:44.164572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 927/1863] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:45.058171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3811e-ef1e-41b8-919c-5854e63a9b63 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:45.059766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-54b3811e-ef1e-41b8-919c-5854e63a9b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:45.072822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3811e-ef1e-41b8-919c-5854e63a9b63 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:45.077601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:45.079857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:45.080324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:45.080800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] PATH_INFO: `/v3/projects/c1e18c502f8a49cda53ffd192d8d4485` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:45.082279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e18c502f8a49cda53ffd192d8d4485)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:45.084335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:45.318669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e18c502f8a49cda53ffd192d8d4485', 'request_id': 'req-e2ad8e50-1de6-4b12-8631-a84cc4c22446'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:45.324341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e18c502f8a49cda53ffd192d8d4485', 'request_id': 'req-e2ad8e50-1de6-4b12-8631-a84cc4c22446'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:45.330785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e18c502f8a49cda53ffd192d8d4485', 'request_id': 'req-e2ad8e50-1de6-4b12-8631-a84cc4c22446'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:45.412577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] The token cache is being invalidate because project c1e18c502f8a49cda53ffd192d8d4485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:45.412903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:45.413533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:45.415149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 937/1864] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:45 2022] DELETE /identity/v3/projects/c1e18c502f8a49cda53ffd192d8d4485 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:45.419178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5b42a-83a6-4995-8809-2724055325a4 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:45.429591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5b42a-83a6-4995-8809-2724055325a4 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:45.485884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:45.486705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:45.486990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:45.487269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] PATH_INFO: `/v3/users/1b7c9c577e2c4ed5b423a42d844ac585` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:45.488454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b7c9c577e2c4ed5b423a42d844ac585)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:45.490430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:45.732323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b7c9c577e2c4ed5b423a42d844ac585'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:45.734888 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:45.745461 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b7c9c577e2c4ed5b423a42d844ac585'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:45.745461 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b7c9c577e2c4ed5b423a42d844ac585'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:45.767346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b7c9c577e2c4ed5b423a42d844ac585'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:45.773391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b7c9c577e2c4ed5b423a42d844ac585'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:45.893171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 928/1865] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:45 2022] DELETE /identity/v3/users/1b7c9c577e2c4ed5b423a42d844ac585 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:46.389699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:46.391334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:46.406233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad8e50-1de6-4b12-8631-a84cc4c22446 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:46.499309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:46.500471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:46.500896 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:46.501180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] PATH_INFO: `/v3/projects/b96d9d2bad344232bc797b35a7440a89` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:46.502641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96d9d2bad344232bc797b35a7440a89)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:46.505295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:46.649730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96d9d2bad344232bc797b35a7440a89', 'request_id': 'req-4a9d4e19-c25e-491d-a059-8e27dfadf817'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:46.655669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96d9d2bad344232bc797b35a7440a89', 'request_id': 'req-4a9d4e19-c25e-491d-a059-8e27dfadf817'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:46.662844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96d9d2bad344232bc797b35a7440a89', 'request_id': 'req-4a9d4e19-c25e-491d-a059-8e27dfadf817'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:46.785596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:46.790657 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] The token cache is being invalidate because project b96d9d2bad344232bc797b35a7440a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:46.791027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:46.791725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:46.794074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 938/1866] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:46 2022] DELETE /identity/v3/projects/b96d9d2bad344232bc797b35a7440a89 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:46.798099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:46.799764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:46.813429 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c06e68-a324-4f66-a40b-54bad37441fa admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:46.906119 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:46.907135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:46.907503 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:46.907870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] PATH_INFO: `/v3/users/67497d833f154678b815c521427164fd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:46.909181 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67497d833f154678b815c521427164fd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:46.911714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:47.157083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67497d833f154678b815c521427164fd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:47.172977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67497d833f154678b815c521427164fd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:47.173706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67497d833f154678b815c521427164fd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:47.199147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67497d833f154678b815c521427164fd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:47.200175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-3a6f8d28-6706-4d59-8dc5-bb616d718b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67497d833f154678b815c521427164fd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:47.309134 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 929/1867] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:46 2022] DELETE /identity/v3/users/67497d833f154678b815c521427164fd => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:47.770951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:47.773111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:47.790644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d4e19-c25e-491d-a059-8e27dfadf817 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:47.848957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:47.849814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:47.850087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:47.850367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] PATH_INFO: `/v3/projects/7196748c50c74591bb23a9f92b4e4e36` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:47.851680 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7196748c50c74591bb23a9f92b4e4e36)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:47.853613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:48.048034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7196748c50c74591bb23a9f92b4e4e36', 'request_id': 'req-fedc34b7-2795-4fe8-b02d-258acf8ce93e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:48.076525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7196748c50c74591bb23a9f92b4e4e36', 'request_id': 'req-fedc34b7-2795-4fe8-b02d-258acf8ce93e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:48.090617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7196748c50c74591bb23a9f92b4e4e36', 'request_id': 'req-fedc34b7-2795-4fe8-b02d-258acf8ce93e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:48.172458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] The token cache is being invalidate because project 7196748c50c74591bb23a9f92b4e4e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:48.172458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:48.172458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fedc34b7-2795-4fe8-b02d-258acf8ce93e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:48.172458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 939/1868] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:47 2022] DELETE /identity/v3/projects/7196748c50c74591bb23a9f92b4e4e36 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:48.177705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86790e3-ac84-4b46-b917-203c8c801035 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:48.178202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86790e3-ac84-4b46-b917-203c8c801035 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:48.178325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86790e3-ac84-4b46-b917-203c8c801035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:48.196810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d86790e3-ac84-4b46-b917-203c8c801035 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:48.198364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d86790e3-ac84-4b46-b917-203c8c801035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:48.244148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:48.265338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 930/1869] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:48.272928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409c45e-cb47-4f35-b13b-047a23c80b9f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:48.273312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409c45e-cb47-4f35-b13b-047a23c80b9f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:48.273837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409c45e-cb47-4f35-b13b-047a23c80b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:48.297831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f409c45e-cb47-4f35-b13b-047a23c80b9f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:48.299287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f409c45e-cb47-4f35-b13b-047a23c80b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:48.311161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 940/1870] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:48.315585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86790e3-ac84-4b46-b917-203c8c801035 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:48.326322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86790e3-ac84-4b46-b917-203c8c801035 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:48.331065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c203b50-46a2-4f4c-bb23-e419eac1c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:48.331729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c203b50-46a2-4f4c-bb23-e419eac1c1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:48.332015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c203b50-46a2-4f4c-bb23-e419eac1c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:48.332293 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c203b50-46a2-4f4c-bb23-e419eac1c1d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:48.333236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c203b50-46a2-4f4c-bb23-e419eac1c1d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:09:48.333569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c203b50-46a2-4f4c-bb23-e419eac1c1d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:48.335392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c203b50-46a2-4f4c-bb23-e419eac1c1d7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:48.347550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 931/1871] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:09:48.352731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409c45e-cb47-4f35-b13b-047a23c80b9f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:48.362877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409c45e-cb47-4f35-b13b-047a23c80b9f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:48.368379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1949cd-749b-418a-9663-f5edba2b648f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:48.369298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1949cd-749b-418a-9663-f5edba2b648f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:48.369682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1949cd-749b-418a-9663-f5edba2b648f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:48.370057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1949cd-749b-418a-9663-f5edba2b648f admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:48.370878 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1949cd-749b-418a-9663-f5edba2b648f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:48.372918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1949cd-749b-418a-9663-f5edba2b648f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:48.479204 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 941/1872] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:48 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:09:48.483752 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c203b50-46a2-4f4c-bb23-e419eac1c1d7 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:48.496502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c203b50-46a2-4f4c-bb23-e419eac1c1d7 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:48.557546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c13a5-8ee2-4203-8454-36c26b5e63b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:48.558231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c13a5-8ee2-4203-8454-36c26b5e63b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:48.558542 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c13a5-8ee2-4203-8454-36c26b5e63b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:48.558821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c13a5-8ee2-4203-8454-36c26b5e63b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:48.559516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c13a5-8ee2-4203-8454-36c26b5e63b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:48.561547 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c13a5-8ee2-4203-8454-36c26b5e63b1 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:48.666912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 932/1873] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:48 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:48.671220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1949cd-749b-418a-9663-f5edba2b648f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:48.682658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1949cd-749b-418a-9663-f5edba2b648f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:48.687795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d847de-64b5-4a89-bc5f-f0951b91653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:48.688639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d847de-64b5-4a89-bc5f-f0951b91653a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:48.688953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d847de-64b5-4a89-bc5f-f0951b91653a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:48.689238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d847de-64b5-4a89-bc5f-f0951b91653a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:48.689985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d847de-64b5-4a89-bc5f-f0951b91653a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:48.691614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d847de-64b5-4a89-bc5f-f0951b91653a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:48.704440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 942/1874] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:09:48.708198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c13a5-8ee2-4203-8454-36c26b5e63b1 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:48.720507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c13a5-8ee2-4203-8454-36c26b5e63b1 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:48.725982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1831899-451e-4b5c-ae2e-746c3b014347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:48.726185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1831899-451e-4b5c-ae2e-746c3b014347 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:48.726448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1831899-451e-4b5c-ae2e-746c3b014347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:48.726756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1831899-451e-4b5c-ae2e-746c3b014347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:48.727493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1831899-451e-4b5c-ae2e-746c3b014347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:48.729245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1831899-451e-4b5c-ae2e-746c3b014347 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:48.740750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 933/1875] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:48.744921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d847de-64b5-4a89-bc5f-f0951b91653a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:48.756364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d847de-64b5-4a89-bc5f-f0951b91653a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:48.761296 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff64450-3395-44a5-848e-4648b7866280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:48.762191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff64450-3395-44a5-848e-4648b7866280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:48.762474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff64450-3395-44a5-848e-4648b7866280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:48.762748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff64450-3395-44a5-848e-4648b7866280 admin admin] PATH_INFO: `/v3/projects/073e4b7f44b74d6ca768408b99445f88/users/a21409773935432e9be918920d6b86d9/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:48.786982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff64450-3395-44a5-848e-4648b7866280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=073e4b7f44b74d6ca768408b99445f88, user_id=a21409773935432e9be918920d6b86d9, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:48.789690 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff64450-3395-44a5-848e-4648b7866280 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:48.929373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 943/1876] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:48 2022] PUT /identity/v3/projects/073e4b7f44b74d6ca768408b99445f88/users/a21409773935432e9be918920d6b86d9/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:48.943077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e4ae7-03b7-467d-ba44-fa7b8de0db5d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:48.943570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e4ae7-03b7-467d-ba44-fa7b8de0db5d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:48.943862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e4ae7-03b7-467d-ba44-fa7b8de0db5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:48.989917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c14e4ae7-03b7-467d-ba44-fa7b8de0db5d None None] MFA Rules not processed for user `a21409773935432e9be918920d6b86d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:48.991173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c14e4ae7-03b7-467d-ba44-fa7b8de0db5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:49.064866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 934/1877] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:09:49.069640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff64450-3395-44a5-848e-4648b7866280 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:49.108645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:49.153020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff64450-3395-44a5-848e-4648b7866280 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:49.158383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bd54d-7545-45ba-b5f4-d24ff013b61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:49.159933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bd54d-7545-45ba-b5f4-d24ff013b61a admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:49.160374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bd54d-7545-45ba-b5f4-d24ff013b61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:49.161362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bd54d-7545-45ba-b5f4-d24ff013b61a admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:49.162225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bd54d-7545-45ba-b5f4-d24ff013b61a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:49.164360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bd54d-7545-45ba-b5f4-d24ff013b61a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:49.309074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 944/1878] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:49 2022] POST /identity/v3/projects => generated 385 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:49.310513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e4ae7-03b7-467d-ba44-fa7b8de0db5d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:49.322014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e4ae7-03b7-467d-ba44-fa7b8de0db5d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:49.367449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:49.385750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6382d2-979e-46f5-b1c3-7b17b457eb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:49.387495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6382d2-979e-46f5-b1c3-7b17b457eb95 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:49.387495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6382d2-979e-46f5-b1c3-7b17b457eb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:49.387495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6382d2-979e-46f5-b1c3-7b17b457eb95 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:49.388280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6382d2-979e-46f5-b1c3-7b17b457eb95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:49.390505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6382d2-979e-46f5-b1c3-7b17b457eb95 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:49.509529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 935/1879] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:49 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:49.514108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bd54d-7545-45ba-b5f4-d24ff013b61a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:49.526158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bd54d-7545-45ba-b5f4-d24ff013b61a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:49.533471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3379f-702b-4161-9728-25165b818dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:49.534485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3379f-702b-4161-9728-25165b818dac admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:49.534979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3379f-702b-4161-9728-25165b818dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:49.535438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3379f-702b-4161-9728-25165b818dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:49.537012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3379f-702b-4161-9728-25165b818dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:49.539470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3379f-702b-4161-9728-25165b818dac admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:49.553101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 945/1880] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:49.557135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6382d2-979e-46f5-b1c3-7b17b457eb95 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:49.569900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6382d2-979e-46f5-b1c3-7b17b457eb95 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:49.575327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb142b-3633-4945-9135-45e8c11f3c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:49.576656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb142b-3633-4945-9135-45e8c11f3c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:49.577075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb142b-3633-4945-9135-45e8c11f3c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:49.583244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb142b-3633-4945-9135-45e8c11f3c8e admin admin] PATH_INFO: `/v3/projects/b444cf5694d24f549e983b70b61dfb61/users/a8f1000d5ca1428b8091e6bc010d0041/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:49.596864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb142b-3633-4945-9135-45e8c11f3c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b444cf5694d24f549e983b70b61dfb61, user_id=a8f1000d5ca1428b8091e6bc010d0041, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:49.601332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb142b-3633-4945-9135-45e8c11f3c8e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:49.718968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 936/1881] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:49 2022] PUT /identity/v3/projects/b444cf5694d24f549e983b70b61dfb61/users/a8f1000d5ca1428b8091e6bc010d0041/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:49.723222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3379f-702b-4161-9728-25165b818dac admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:49.739045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3379f-702b-4161-9728-25165b818dac admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:49.840633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e53ead-a1d6-473b-a18a-fe10767426ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:49.841409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53ead-a1d6-473b-a18a-fe10767426ed admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:49.841696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53ead-a1d6-473b-a18a-fe10767426ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:49.841972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e53ead-a1d6-473b-a18a-fe10767426ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:49.842707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e53ead-a1d6-473b-a18a-fe10767426ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:49.844408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e53ead-a1d6-473b-a18a-fe10767426ed admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:49.855687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 946/1882] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:49.860164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb142b-3633-4945-9135-45e8c11f3c8e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:49.881619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb142b-3633-4945-9135-45e8c11f3c8e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:49.890075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce140b7-a755-4223-8ffc-fa12037040f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:49.890983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce140b7-a755-4223-8ffc-fa12037040f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:49.891271 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce140b7-a755-4223-8ffc-fa12037040f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:49.891544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce140b7-a755-4223-8ffc-fa12037040f8 admin admin] PATH_INFO: `/v3/projects/b444cf5694d24f549e983b70b61dfb61/users/a8f1000d5ca1428b8091e6bc010d0041/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:49.896289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce140b7-a755-4223-8ffc-fa12037040f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b444cf5694d24f549e983b70b61dfb61, user_id=a8f1000d5ca1428b8091e6bc010d0041, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:49.900014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce140b7-a755-4223-8ffc-fa12037040f8 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:49.948592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 937/1883] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:49 2022] PUT /identity/v3/projects/b444cf5694d24f549e983b70b61dfb61/users/a8f1000d5ca1428b8091e6bc010d0041/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:49.949452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d11e7a-43bf-4886-918e-78b50fe77acf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:49.949840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d11e7a-43bf-4886-918e-78b50fe77acf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:49.950318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d11e7a-43bf-4886-918e-78b50fe77acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:49.995524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f6d11e7a-43bf-4886-918e-78b50fe77acf None None] MFA Rules not processed for user `a8f1000d5ca1428b8091e6bc010d0041`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:49.997062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6d11e7a-43bf-4886-918e-78b50fe77acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:50.111306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 947/1884] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:50.388455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce140b7-a755-4223-8ffc-fa12037040f8 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:50.390251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-dce140b7-a755-4223-8ffc-fa12037040f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:50.487290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce140b7-a755-4223-8ffc-fa12037040f8 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:50.492247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d6b8f-291e-4f8c-8c82-af31b74cba1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:50.493764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d6b8f-291e-4f8c-8c82-af31b74cba1b service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:50.494230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d6b8f-291e-4f8c-8c82-af31b74cba1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:50.494651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d6b8f-291e-4f8c-8c82-af31b74cba1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:50.507829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d6b8f-291e-4f8c-8c82-af31b74cba1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:50.509662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d6b8f-291e-4f8c-8c82-af31b74cba1b service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:50.541015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 938/1885] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:52.229362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d11e7a-43bf-4886-918e-78b50fe77acf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:52.281383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:52.361782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d11e7a-43bf-4886-918e-78b50fe77acf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:52.367157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:52.368103 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:52.368477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:52.368855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] PATH_INFO: `/v3/users/a21409773935432e9be918920d6b86d9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:52.370257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a21409773935432e9be918920d6b86d9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:52.372393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:52.618514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a21409773935432e9be918920d6b86d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:52.632665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a21409773935432e9be918920d6b86d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:52.633792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a21409773935432e9be918920d6b86d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:52.635120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a21409773935432e9be918920d6b86d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:52.651496 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a21409773935432e9be918920d6b86d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:52.754930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 948/1886] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:52 2022] DELETE /identity/v3/users/a21409773935432e9be918920d6b86d9 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:52.758405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c987a-f47f-4df2-bf06-1712f6e9df86 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:52.758839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c987a-f47f-4df2-bf06-1712f6e9df86 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:52.759170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c987a-f47f-4df2-bf06-1712f6e9df86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:52.780859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-a59c987a-f47f-4df2-bf06-1712f6e9df86 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:52.782251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a59c987a-f47f-4df2-bf06-1712f6e9df86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:52.864208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 939/1887] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:52.874598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:52.876386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:52.952432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9cc37-d407-44a4-a090-affb77748ea4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:52.965644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80253d42-b00d-4124-a8ca-1552c2a38c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:52.966497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80253d42-b00d-4124-a8ca-1552c2a38c2d service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:52.966849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80253d42-b00d-4124-a8ca-1552c2a38c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:52.967195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80253d42-b00d-4124-a8ca-1552c2a38c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:52.978739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80253d42-b00d-4124-a8ca-1552c2a38c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:52.980210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80253d42-b00d-4124-a8ca-1552c2a38c2d service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:52.996037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 949/1888] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:53.435841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c987a-f47f-4df2-bf06-1712f6e9df86 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:53.437478 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a59c987a-f47f-4df2-bf06-1712f6e9df86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:53.453598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c987a-f47f-4df2-bf06-1712f6e9df86 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:53.461368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:53.462843 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:53.463400 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:53.463750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] PATH_INFO: `/v3/projects/073e4b7f44b74d6ca768408b99445f88` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:53.465151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=073e4b7f44b74d6ca768408b99445f88)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:53.467262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:53.559136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:53.571040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '073e4b7f44b74d6ca768408b99445f88', 'request_id': 'req-ea67159d-3e70-4e89-a326-daed302bec7c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:53.578426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '073e4b7f44b74d6ca768408b99445f88', 'request_id': 'req-ea67159d-3e70-4e89-a326-daed302bec7c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:53.688365 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '073e4b7f44b74d6ca768408b99445f88', 'request_id': 'req-ea67159d-3e70-4e89-a326-daed302bec7c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:53.698091 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] The token cache is being invalidate because project 073e4b7f44b74d6ca768408b99445f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:53.698385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:53.699030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:53.700700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 940/1889] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:53 2022] DELETE /identity/v3/projects/073e4b7f44b74d6ca768408b99445f88 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:53.704861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80253d42-b00d-4124-a8ca-1552c2a38c2d service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:53.716115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80253d42-b00d-4124-a8ca-1552c2a38c2d service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:53.762570 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:53.779901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:53.780851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:53.781094 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:53.781368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] PATH_INFO: `/v3/users/a8f1000d5ca1428b8091e6bc010d0041` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:53.783169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f1000d5ca1428b8091e6bc010d0041)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:53.784534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:54.027430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f1000d5ca1428b8091e6bc010d0041'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:54.047313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f1000d5ca1428b8091e6bc010d0041'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:54.048094 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f1000d5ca1428b8091e6bc010d0041'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:54.048974 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f1000d5ca1428b8091e6bc010d0041'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:54.064040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b6f75e2e-ea27-4c77-b288-2979d779f58a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f1000d5ca1428b8091e6bc010d0041'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:54.143944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 950/1890] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:53 2022] DELETE /identity/v3/users/a8f1000d5ca1428b8091e6bc010d0041 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:54.795361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:54.797024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:54.820454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67159d-3e70-4e89-a326-daed302bec7c admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:54.898148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:54.899039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:54.899327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:54.899608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] PATH_INFO: `/v3/projects/b444cf5694d24f549e983b70b61dfb61` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:54.900960 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b444cf5694d24f549e983b70b61dfb61)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:54.902771 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:55.010501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b444cf5694d24f549e983b70b61dfb61', 'request_id': 'req-a0841921-83b1-42a8-a810-8ee8ee77c815'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:55.017167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b444cf5694d24f549e983b70b61dfb61', 'request_id': 'req-a0841921-83b1-42a8-a810-8ee8ee77c815'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:55.083788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b444cf5694d24f549e983b70b61dfb61', 'request_id': 'req-a0841921-83b1-42a8-a810-8ee8ee77c815'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:55.110046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] The token cache is being invalidate because project b444cf5694d24f549e983b70b61dfb61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:55.110721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:55.112466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a0841921-83b1-42a8-a810-8ee8ee77c815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:55.118414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 941/1891] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:54 2022] DELETE /identity/v3/projects/b444cf5694d24f549e983b70b61dfb61 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:55.124960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97719d39-76e5-4a47-93fe-261224f0d1e1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.125452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97719d39-76e5-4a47-93fe-261224f0d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.125607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97719d39-76e5-4a47-93fe-261224f0d1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.144660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee457d5f-a476-4713-8391-96f9c23c9f58 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.147586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee457d5f-a476-4713-8391-96f9c23c9f58 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.148066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee457d5f-a476-4713-8391-96f9c23c9f58 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.152121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 942/1892] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:09:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:09:55.160007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-97719d39-76e5-4a47-93fe-261224f0d1e1 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:55.162158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-97719d39-76e5-4a47-93fe-261224f0d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:55.189654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f759-f9c4-4d0d-af32-66ff8457669a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.190314 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f759-f9c4-4d0d-af32-66ff8457669a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.190777 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f759-f9c4-4d0d-af32-66ff8457669a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.212083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d4f5f759-f9c4-4d0d-af32-66ff8457669a None None] MFA Rules not processed for user `75b7f541993b4a7c915c9e6d440e2674`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:55.213561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d4f5f759-f9c4-4d0d-af32-66ff8457669a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:55.240045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 951/1893] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:55.249892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baf1a7-266c-4d35-bbec-fc457beeed3a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.250295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baf1a7-266c-4d35-bbec-fc457beeed3a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.250743 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baf1a7-266c-4d35-bbec-fc457beeed3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.293702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d7baf1a7-266c-4d35-bbec-fc457beeed3a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:55.296134 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d7baf1a7-266c-4d35-bbec-fc457beeed3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:55.318488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:55.324063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 952/1894] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:55.328673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baf1a7-266c-4d35-bbec-fc457beeed3a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:55.341175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baf1a7-266c-4d35-bbec-fc457beeed3a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:55.347803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 943/1895] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:09:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:09:55.352545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdaa77-5446-4b62-9608-9b9779984474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:55.352545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdaa77-5446-4b62-9608-9b9779984474 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.352545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdaa77-5446-4b62-9608-9b9779984474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.352545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdaa77-5446-4b62-9608-9b9779984474 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.352934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fdaa77-5446-4b62-9608-9b9779984474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:09:55.352934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fdaa77-5446-4b62-9608-9b9779984474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:55.352934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fdaa77-5446-4b62-9608-9b9779984474 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:55.360055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5f759-f9c4-4d0d-af32-66ff8457669a None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:55.361667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d4f5f759-f9c4-4d0d-af32-66ff8457669a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:55.368029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 953/1896] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:09:55.372376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdaa77-5446-4b62-9608-9b9779984474 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:55.376576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5f759-f9c4-4d0d-af32-66ff8457669a None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:55.383877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdaa77-5446-4b62-9608-9b9779984474 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:55.384742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34027dd7-1fd1-42e6-be9e-1957aa257f42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:55.385666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34027dd7-1fd1-42e6-be9e-1957aa257f42 service placement] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.385666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34027dd7-1fd1-42e6-be9e-1957aa257f42 service placement] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.385949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34027dd7-1fd1-42e6-be9e-1957aa257f42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.389472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc070ad6-5795-4058-a188-70f4cb7e7f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:55.390266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc070ad6-5795-4058-a188-70f4cb7e7f50 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.390617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc070ad6-5795-4058-a188-70f4cb7e7f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.390960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc070ad6-5795-4058-a188-70f4cb7e7f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.391878 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc070ad6-5795-4058-a188-70f4cb7e7f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:55.393937 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc070ad6-5795-4058-a188-70f4cb7e7f50 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:55.396420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34027dd7-1fd1-42e6-be9e-1957aa257f42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:55.398201 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34027dd7-1fd1-42e6-be9e-1957aa257f42 service placement] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:55.409691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 944/1897] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:09:55.543984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 954/1898] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:55 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:09:55.548141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34027dd7-1fd1-42e6-be9e-1957aa257f42 service placement] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:55.559174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34027dd7-1fd1-42e6-be9e-1957aa257f42 service placement] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:55.618997 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50973105-971c-4b43-8eba-47a2009fe515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:55.619733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50973105-971c-4b43-8eba-47a2009fe515 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.621843 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50973105-971c-4b43-8eba-47a2009fe515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.621843 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50973105-971c-4b43-8eba-47a2009fe515 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.621843 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50973105-971c-4b43-8eba-47a2009fe515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:55.624033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50973105-971c-4b43-8eba-47a2009fe515 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:55.723519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 945/1899] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:55 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:55.724730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc070ad6-5795-4058-a188-70f4cb7e7f50 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:55.735393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc070ad6-5795-4058-a188-70f4cb7e7f50 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:55.740800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9269c9-1170-45fe-b088-702829d057f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:55.741471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9269c9-1170-45fe-b088-702829d057f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.741758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9269c9-1170-45fe-b088-702829d057f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.742042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9269c9-1170-45fe-b088-702829d057f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.742759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9269c9-1170-45fe-b088-702829d057f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:55.744366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9269c9-1170-45fe-b088-702829d057f7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:55.754969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 955/1900] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:55.759023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50973105-971c-4b43-8eba-47a2009fe515 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:55.769443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50973105-971c-4b43-8eba-47a2009fe515 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:55.774100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200c1cd-cbf5-47d1-a125-6b270becab43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:55.774871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200c1cd-cbf5-47d1-a125-6b270becab43 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.775192 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200c1cd-cbf5-47d1-a125-6b270becab43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.775527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200c1cd-cbf5-47d1-a125-6b270becab43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.776307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200c1cd-cbf5-47d1-a125-6b270becab43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:55.778023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200c1cd-cbf5-47d1-a125-6b270becab43 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:55.788624 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 946/1901] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:55.792841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9269c9-1170-45fe-b088-702829d057f7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:55.803735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9269c9-1170-45fe-b088-702829d057f7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:55.810774 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a6085-a974-433d-a2a9-0665915430a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:55.811661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a6085-a974-433d-a2a9-0665915430a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.811946 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a6085-a974-433d-a2a9-0665915430a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.812219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a6085-a974-433d-a2a9-0665915430a6 admin admin] PATH_INFO: `/v3/projects/da19a33ebcf44749a258d7aa3ca9b9b6/users/abd13551affb4d6c98b4e071634eea05/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.821184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:55.827951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a6085-a974-433d-a2a9-0665915430a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da19a33ebcf44749a258d7aa3ca9b9b6, user_id=abd13551affb4d6c98b4e071634eea05, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:55.830694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a6085-a974-433d-a2a9-0665915430a6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:55.912386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 956/1902] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:55 2022] PUT /identity/v3/projects/da19a33ebcf44749a258d7aa3ca9b9b6/users/abd13551affb4d6c98b4e071634eea05/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:55.923387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2dd12-dd63-401a-ac43-e8ed6b2dd13f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:55.924036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2dd12-dd63-401a-ac43-e8ed6b2dd13f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:55.924224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2dd12-dd63-401a-ac43-e8ed6b2dd13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:55.953816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c2c2dd12-dd63-401a-ac43-e8ed6b2dd13f None None] MFA Rules not processed for user `abd13551affb4d6c98b4e071634eea05`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:55.955143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c2c2dd12-dd63-401a-ac43-e8ed6b2dd13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:56.024713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a6085-a974-433d-a2a9-0665915430a6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:56.024896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 947/1903] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:56.033589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bbcd7-0f35-4377-8459-e036c65a890e None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.033757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bbcd7-0f35-4377-8459-e036c65a890e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.034066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bbcd7-0f35-4377-8459-e036c65a890e None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.036058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 948/1904] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 16:09:56.047628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917c24f-c991-477a-8fbd-ce153bbad456 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.048710 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917c24f-c991-477a-8fbd-ce153bbad456 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.049312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917c24f-c991-477a-8fbd-ce153bbad456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.069789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e917c24f-c991-477a-8fbd-ce153bbad456 None None] MFA Rules not processed for user `75b7f541993b4a7c915c9e6d440e2674`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:56.070932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e917c24f-c991-477a-8fbd-ce153bbad456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:56.091418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a6085-a974-433d-a2a9-0665915430a6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:56.096021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96426aae-42fc-4e03-9ede-7eebf95dad92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:56.099149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96426aae-42fc-4e03-9ede-7eebf95dad92 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.099149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96426aae-42fc-4e03-9ede-7eebf95dad92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.099149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96426aae-42fc-4e03-9ede-7eebf95dad92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.099149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96426aae-42fc-4e03-9ede-7eebf95dad92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:56.100446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96426aae-42fc-4e03-9ede-7eebf95dad92 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:56.117662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:56.144142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 949/1905] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:09:56.145774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917c24f-c991-477a-8fbd-ce153bbad456 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:56.147516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e917c24f-c991-477a-8fbd-ce153bbad456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:56.163572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917c24f-c991-477a-8fbd-ce153bbad456 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:56.168442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:56.169374 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.169653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.169923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] PATH_INFO: `/v3/users/66375274e9fb44d39936ebf64af69152` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.171108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=66375274e9fb44d39936ebf64af69152)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:56.173101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:56.184719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 957/1906] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:56 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:09:56.188340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96426aae-42fc-4e03-9ede-7eebf95dad92 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:56.255140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96426aae-42fc-4e03-9ede-7eebf95dad92 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:56.260362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaf97d-a44a-47b0-afa5-75216bc50ed7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:56.261028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaf97d-a44a-47b0-afa5-75216bc50ed7 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.261312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaf97d-a44a-47b0-afa5-75216bc50ed7 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.261585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaf97d-a44a-47b0-afa5-75216bc50ed7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.270531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaf97d-a44a-47b0-afa5-75216bc50ed7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:56.272266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaf97d-a44a-47b0-afa5-75216bc50ed7 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:56.282725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 958/1907] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:09:56.283545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaf97d-a44a-47b0-afa5-75216bc50ed7 service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:56.361349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaf97d-a44a-47b0-afa5-75216bc50ed7 service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:56.365948 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254adaa3-9852-4b1e-954b-602f89a25f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:56.366619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254adaa3-9852-4b1e-954b-602f89a25f2d admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.366904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254adaa3-9852-4b1e-954b-602f89a25f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.367182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254adaa3-9852-4b1e-954b-602f89a25f2d admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.367876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254adaa3-9852-4b1e-954b-602f89a25f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:56.369824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254adaa3-9852-4b1e-954b-602f89a25f2d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:56.377254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:56.396544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66375274e9fb44d39936ebf64af69152'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:56.403430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66375274e9fb44d39936ebf64af69152'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:56.404091 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66375274e9fb44d39936ebf64af69152'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:56.416408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66375274e9fb44d39936ebf64af69152'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:56.417131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66375274e9fb44d39936ebf64af69152'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:56.454743 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 959/1908] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:56 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:56.459546 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254adaa3-9852-4b1e-954b-602f89a25f2d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:56.461354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-254adaa3-9852-4b1e-954b-602f89a25f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:56.473969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254adaa3-9852-4b1e-954b-602f89a25f2d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:56.479316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4ea88-2d1a-4bdb-98d2-27148ba1170b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:56.480255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4ea88-2d1a-4bdb-98d2-27148ba1170b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.480717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4ea88-2d1a-4bdb-98d2-27148ba1170b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.481264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4ea88-2d1a-4bdb-98d2-27148ba1170b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.482227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4ea88-2d1a-4bdb-98d2-27148ba1170b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:56.484580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4ea88-2d1a-4bdb-98d2-27148ba1170b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:56.501358 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 960/1909] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:56.501358 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4ea88-2d1a-4bdb-98d2-27148ba1170b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:56.507474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 950/1910] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:56 2022] DELETE /identity/v3/users/66375274e9fb44d39936ebf64af69152 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:56.512702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4ea88-2d1a-4bdb-98d2-27148ba1170b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:56.573373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199a08d-c9fb-437b-b623-a9c7cc1b7e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:56.574425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a08d-c9fb-437b-b623-a9c7cc1b7e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.574678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a08d-c9fb-437b-b623-a9c7cc1b7e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.574910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199a08d-c9fb-437b-b623-a9c7cc1b7e49 admin admin] PATH_INFO: `/v3/projects/2cc069310621438f9eaa51f619ef52ef/users/e270a20326834c0c962f267bcb5708f8/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.590827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199a08d-c9fb-437b-b623-a9c7cc1b7e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc069310621438f9eaa51f619ef52ef, user_id=e270a20326834c0c962f267bcb5708f8, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:56.593549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199a08d-c9fb-437b-b623-a9c7cc1b7e49 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:56.661922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 961/1911] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:56 2022] PUT /identity/v3/projects/2cc069310621438f9eaa51f619ef52ef/users/e270a20326834c0c962f267bcb5708f8/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:09:56.666308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:56.676611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0cb56-9362-4ae6-b821-1a513c5afe7d admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:56.731708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea1de8-b40a-4784-b076-f6aa31abbc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:56.732370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea1de8-b40a-4784-b076-f6aa31abbc7d admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.732653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea1de8-b40a-4784-b076-f6aa31abbc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.732959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea1de8-b40a-4784-b076-f6aa31abbc7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.733677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea1de8-b40a-4784-b076-f6aa31abbc7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:56.735538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea1de8-b40a-4784-b076-f6aa31abbc7d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:56.746255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 951/1912] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:56.750325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199a08d-c9fb-437b-b623-a9c7cc1b7e49 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:56.762368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199a08d-c9fb-437b-b623-a9c7cc1b7e49 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:56.767746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea1de8-b40a-4784-b076-f6aa31abbc7d admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:56.767872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5120e3-d66b-4673-ad00-5dbf1113ed78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:56.768724 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5120e3-d66b-4673-ad00-5dbf1113ed78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.769037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5120e3-d66b-4673-ad00-5dbf1113ed78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.769284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5120e3-d66b-4673-ad00-5dbf1113ed78 admin admin] PATH_INFO: `/v3/projects/2cc069310621438f9eaa51f619ef52ef/users/e270a20326834c0c962f267bcb5708f8/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.770303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-10ea1de8-b40a-4784-b076-f6aa31abbc7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:56.771340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5120e3-d66b-4673-ad00-5dbf1113ed78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc069310621438f9eaa51f619ef52ef, user_id=e270a20326834c0c962f267bcb5708f8, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:56.774106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5120e3-d66b-4673-ad00-5dbf1113ed78 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:56.784912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea1de8-b40a-4784-b076-f6aa31abbc7d admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:56.787711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 962/1913] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:56 2022] PUT /identity/v3/projects/2cc069310621438f9eaa51f619ef52ef/users/e270a20326834c0c962f267bcb5708f8/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:56.797059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457e4d5-ef05-43b5-8cac-70603811f377 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.797304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457e4d5-ef05-43b5-8cac-70603811f377 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.797585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457e4d5-ef05-43b5-8cac-70603811f377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.829420 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e457e4d5-ef05-43b5-8cac-70603811f377 None None] MFA Rules not processed for user `e270a20326834c0c962f267bcb5708f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:56.830566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e457e4d5-ef05-43b5-8cac-70603811f377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:56.840751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:56.854457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:56.855404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:56.855689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:56.855961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] PATH_INFO: `/v3/projects/bbaa897514934520bd6eb61aaa872290` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:56.857190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbaa897514934520bd6eb61aaa872290)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:56.859899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:56.903945 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 963/1914] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:09:56.914731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457e4d5-ef05-43b5-8cac-70603811f377 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:56.916378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e457e4d5-ef05-43b5-8cac-70603811f377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:56.957779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbaa897514934520bd6eb61aaa872290', 'request_id': 'req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:56.959683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:56.966115 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbaa897514934520bd6eb61aaa872290', 'request_id': 'req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:56.978611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457e4d5-ef05-43b5-8cac-70603811f377 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:57.029025 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b7efd-6140-4253-a278-85cb1373b832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:57.029677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b7efd-6140-4253-a278-85cb1373b832 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:57.029954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b7efd-6140-4253-a278-85cb1373b832 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:57.030227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b7efd-6140-4253-a278-85cb1373b832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:57.032012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-131b7efd-6140-4253-a278-85cb1373b832 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:57.039258 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbaa897514934520bd6eb61aaa872290', 'request_id': 'req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:57.052835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] The token cache is being invalidate because project bbaa897514934520bd6eb61aaa872290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:57.053105 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:57.053777 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:57.058251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 952/1915] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:56 2022] DELETE /identity/v3/projects/bbaa897514934520bd6eb61aaa872290 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:57.062338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:57.064621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:57.105965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b7efd-6140-4253-a278-85cb1373b832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:57.107490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b7efd-6140-4253-a278-85cb1373b832 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:57.118297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 964/1916] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:09:57.136210 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965eafa-8bcc-4170-8ab6-c48e0d64eba8 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:57.141257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:57.143921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:57.143921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:57.143921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] PATH_INFO: `/v3/users/e6ec9086552b4b26a34c892ded10f739` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:57.144133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ec9086552b4b26a34c892ded10f739)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:57.146568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:57.405116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ec9086552b4b26a34c892ded10f739'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:57.412287 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ec9086552b4b26a34c892ded10f739'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:57.412973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ec9086552b4b26a34c892ded10f739'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:57.424242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ec9086552b4b26a34c892ded10f739'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:57.424923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ac31535c-9b66-4511-985d-97dc2697a5c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ec9086552b4b26a34c892ded10f739'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:57.501402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:57.506918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 953/1917] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:09:57 2022] DELETE /identity/v3/users/e6ec9086552b4b26a34c892ded10f739 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:09:57.783591 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b7efd-6140-4253-a278-85cb1373b832 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:57.795187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b7efd-6140-4253-a278-85cb1373b832 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:57.864626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:57.865521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:57.865808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:57.866085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] PATH_INFO: `/v3/projects/f8f4429bcd344015956046f2802063de` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:57.868295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8f4429bcd344015956046f2802063de)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:57.869242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:57.959904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:57.961157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8f4429bcd344015956046f2802063de', 'request_id': 'req-1b2d89ca-79a2-4746-90da-a725a10a4c3b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:57.966440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8f4429bcd344015956046f2802063de', 'request_id': 'req-1b2d89ca-79a2-4746-90da-a725a10a4c3b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:57.972440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8f4429bcd344015956046f2802063de', 'request_id': 'req-1b2d89ca-79a2-4746-90da-a725a10a4c3b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:58.058746 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] The token cache is being invalidate because project f8f4429bcd344015956046f2802063de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:09:58.059035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:58.059638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1b2d89ca-79a2-4746-90da-a725a10a4c3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:09:58.061202 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 965/1918] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:09:57 2022] DELETE /identity/v3/projects/f8f4429bcd344015956046f2802063de => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:58.066058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbae1b-2278-4da8-9b66-8a03d768a1b7 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:58.067047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbae1b-2278-4da8-9b66-8a03d768a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:58.067233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbae1b-2278-4da8-9b66-8a03d768a1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:58.086367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4dcbae1b-2278-4da8-9b66-8a03d768a1b7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:58.092433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4dcbae1b-2278-4da8-9b66-8a03d768a1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:58.150302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 954/1919] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:58.159422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f394d-61da-4ccb-8e31-baeec08f80b6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:58.159747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f394d-61da-4ccb-8e31-baeec08f80b6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:58.160031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f394d-61da-4ccb-8e31-baeec08f80b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:58.178664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-859f394d-61da-4ccb-8e31-baeec08f80b6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:58.179875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-859f394d-61da-4ccb-8e31-baeec08f80b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:58.189682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 966/1920] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:58.193491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbae1b-2278-4da8-9b66-8a03d768a1b7 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:58.203336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbae1b-2278-4da8-9b66-8a03d768a1b7 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:58.208152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784734d2-daac-4e89-81fa-d9e178cd52dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:58.209030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784734d2-daac-4e89-81fa-d9e178cd52dd admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:58.209470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784734d2-daac-4e89-81fa-d9e178cd52dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:58.209894 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784734d2-daac-4e89-81fa-d9e178cd52dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:58.210717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784734d2-daac-4e89-81fa-d9e178cd52dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:09:58.211009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784734d2-daac-4e89-81fa-d9e178cd52dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:58.212751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784734d2-daac-4e89-81fa-d9e178cd52dd admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:58.225058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 955/1921] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:09:58.229348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f394d-61da-4ccb-8e31-baeec08f80b6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:58.239795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f394d-61da-4ccb-8e31-baeec08f80b6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:58.244623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a45b5-4984-4935-8367-4395f65ac001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:58.245384 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a45b5-4984-4935-8367-4395f65ac001 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:58.245673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a45b5-4984-4935-8367-4395f65ac001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:58.245949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a45b5-4984-4935-8367-4395f65ac001 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:58.246656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a45b5-4984-4935-8367-4395f65ac001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:58.249598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a45b5-4984-4935-8367-4395f65ac001 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:58.337636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 967/1922] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:09:58 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:09:58.341712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784734d2-daac-4e89-81fa-d9e178cd52dd admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:58.351452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784734d2-daac-4e89-81fa-d9e178cd52dd admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:58.406267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e75b2-9c5b-4fa4-b1cf-0b0e756f4893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:58.406916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e75b2-9c5b-4fa4-b1cf-0b0e756f4893 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:58.407197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e75b2-9c5b-4fa4-b1cf-0b0e756f4893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:58.407468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e75b2-9c5b-4fa4-b1cf-0b0e756f4893 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:58.408149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e75b2-9c5b-4fa4-b1cf-0b0e756f4893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:58.410066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e75b2-9c5b-4fa4-b1cf-0b0e756f4893 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:58.591159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 956/1923] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:09:58 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:09:58.592314 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a45b5-4984-4935-8367-4395f65ac001 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:58.602937 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a45b5-4984-4935-8367-4395f65ac001 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:58.607506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df70e3-8eeb-4f23-8f19-8e5273b3b9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:58.608171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df70e3-8eeb-4f23-8f19-8e5273b3b9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:58.608460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df70e3-8eeb-4f23-8f19-8e5273b3b9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:58.608731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df70e3-8eeb-4f23-8f19-8e5273b3b9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:58.609750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df70e3-8eeb-4f23-8f19-8e5273b3b9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:58.611340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df70e3-8eeb-4f23-8f19-8e5273b3b9f2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:58.622812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 968/1924] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:09:58.626439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e75b2-9c5b-4fa4-b1cf-0b0e756f4893 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:58.636499 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e75b2-9c5b-4fa4-b1cf-0b0e756f4893 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:58.641099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a599eb-50de-4f5e-b309-2a5bb826f69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:58.641753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a599eb-50de-4f5e-b309-2a5bb826f69c admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:58.642040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a599eb-50de-4f5e-b309-2a5bb826f69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:58.642319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a599eb-50de-4f5e-b309-2a5bb826f69c admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:58.643083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a599eb-50de-4f5e-b309-2a5bb826f69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:58.644658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a599eb-50de-4f5e-b309-2a5bb826f69c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:58.654470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 957/1925] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:09:58.658756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df70e3-8eeb-4f23-8f19-8e5273b3b9f2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:58.669354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df70e3-8eeb-4f23-8f19-8e5273b3b9f2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:58.673887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ec2d0-9670-4db9-ad4e-8d67c0686d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:58.674760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ec2d0-9670-4db9-ad4e-8d67c0686d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:58.675045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ec2d0-9670-4db9-ad4e-8d67c0686d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:58.675320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ec2d0-9670-4db9-ad4e-8d67c0686d25 admin admin] PATH_INFO: `/v3/projects/8ad2c381df664f8599e059a7af1ae673/users/536cba541cc04f4abdad741dd391d9ce/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:58.692419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ec2d0-9670-4db9-ad4e-8d67c0686d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ad2c381df664f8599e059a7af1ae673, user_id=536cba541cc04f4abdad741dd391d9ce, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:58.695307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ec2d0-9670-4db9-ad4e-8d67c0686d25 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:58.971070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 969/1926] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:09:58 2022] PUT /identity/v3/projects/8ad2c381df664f8599e059a7af1ae673/users/536cba541cc04f4abdad741dd391d9ce/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:09:58.982348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60475c2c-b371-4db3-b147-187d1e46ef24 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:58.982683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60475c2c-b371-4db3-b147-187d1e46ef24 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:58.982990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60475c2c-b371-4db3-b147-187d1e46ef24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:59.009817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-60475c2c-b371-4db3-b147-187d1e46ef24 None None] MFA Rules not processed for user `536cba541cc04f4abdad741dd391d9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:09:59.011016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-60475c2c-b371-4db3-b147-187d1e46ef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:59.057690 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:09:59.074506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 958/1927] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:09:59.085843 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ec2d0-9670-4db9-ad4e-8d67c0686d25 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:09:59.087504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-400ec2d0-9670-4db9-ad4e-8d67c0686d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:09:59.143267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ec2d0-9670-4db9-ad4e-8d67c0686d25 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:09:59.147287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16d0e8-c130-4f96-8d1b-65602b8d4e67 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b712bd7d8d4930a3bfc99581be8cb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:09:59.147934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d0e8-c130-4f96-8d1b-65602b8d4e67 service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:09:59.148215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d0e8-c130-4f96-8d1b-65602b8d4e67 service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:09:59.148561 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d0e8-c130-4f96-8d1b-65602b8d4e67 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:09:59.157291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16d0e8-c130-4f96-8d1b-65602b8d4e67 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:09:59.158802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16d0e8-c130-4f96-8d1b-65602b8d4e67 service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:09:59.169484 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 970/1928] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:09:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:06.508358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60475c2c-b371-4db3-b147-187d1e46ef24 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:06.509848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-60475c2c-b371-4db3-b147-187d1e46ef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:06.567645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60475c2c-b371-4db3-b147-187d1e46ef24 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:06.572024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f9d62-55c7-4e89-a2c6-b95d1be1bb2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:06.573049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f9d62-55c7-4e89-a2c6-b95d1be1bb2f service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:06.573333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f9d62-55c7-4e89-a2c6-b95d1be1bb2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:06.573605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f9d62-55c7-4e89-a2c6-b95d1be1bb2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:06.575504 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-021f9d62-55c7-4e89-a2c6-b95d1be1bb2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:06.644740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f9d62-55c7-4e89-a2c6-b95d1be1bb2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:06.646294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f9d62-55c7-4e89-a2c6-b95d1be1bb2f service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:06.656012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 959/1929] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:09.164669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:12.630735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16d0e8-c130-4f96-8d1b-65602b8d4e67 service glance] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:12.632748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-af16d0e8-c130-4f96-8d1b-65602b8d4e67 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:12.698130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16d0e8-c130-4f96-8d1b-65602b8d4e67 service glance] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:12.702716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:12.703600 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:12.703882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:12.704155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] PATH_INFO: `/v3/users/abd13551affb4d6c98b4e071634eea05` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:12.705377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd13551affb4d6c98b4e071634eea05)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:12.707299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:12.916197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd13551affb4d6c98b4e071634eea05'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:12.927681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd13551affb4d6c98b4e071634eea05'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:12.929092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd13551affb4d6c98b4e071634eea05'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:12.929711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd13551affb4d6c98b4e071634eea05'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:12.943456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd13551affb4d6c98b4e071634eea05'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:13.010720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 971/1930] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:12 2022] DELETE /identity/v3/users/abd13551affb4d6c98b4e071634eea05 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:13.015337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3344bf-d094-4b01-ae76-7a6f95900a7f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:13.015862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3344bf-d094-4b01-ae76-7a6f95900a7f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:13.015960 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3344bf-d094-4b01-ae76-7a6f95900a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:13.037044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-ab3344bf-d094-4b01-ae76-7a6f95900a7f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:13.039054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ab3344bf-d094-4b01-ae76-7a6f95900a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:13.101447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 960/1931] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:13.112254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:13.112885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:13.176058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a513285-4a2b-46d3-8b87-1b153dc3a7b8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:13.179778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ff1da-8d1e-4b01-8c59-d4ce98fa2152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:13.181957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ff1da-8d1e-4b01-8c59-d4ce98fa2152 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:13.181957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ff1da-8d1e-4b01-8c59-d4ce98fa2152 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:13.181957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ff1da-8d1e-4b01-8c59-d4ce98fa2152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:13.191094 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ff1da-8d1e-4b01-8c59-d4ce98fa2152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:13.192522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ff1da-8d1e-4b01-8c59-d4ce98fa2152 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:13.202233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 972/1932] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:13.775784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3344bf-d094-4b01-ae76-7a6f95900a7f None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:13.777344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ab3344bf-d094-4b01-ae76-7a6f95900a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:13.792999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3344bf-d094-4b01-ae76-7a6f95900a7f None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:13.798310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:13.799176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:13.799639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:13.800053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] PATH_INFO: `/v3/projects/da19a33ebcf44749a258d7aa3ca9b9b6` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:13.801305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da19a33ebcf44749a258d7aa3ca9b9b6)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:13.803648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:13.920108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:13.931657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da19a33ebcf44749a258d7aa3ca9b9b6', 'request_id': 'req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:13.938863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da19a33ebcf44749a258d7aa3ca9b9b6', 'request_id': 'req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:14.010602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da19a33ebcf44749a258d7aa3ca9b9b6', 'request_id': 'req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:14.025596 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] The token cache is being invalidate because project da19a33ebcf44749a258d7aa3ca9b9b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:14.026272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:14.026742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:14.032691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ff1da-8d1e-4b01-8c59-d4ce98fa2152 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:14.034066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 961/1933] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:13 2022] DELETE /identity/v3/projects/da19a33ebcf44749a258d7aa3ca9b9b6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:14.045210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ff1da-8d1e-4b01-8c59-d4ce98fa2152 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:14.107465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:14.120836 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:14.121892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:14.122403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:14.122753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] PATH_INFO: `/v3/users/e270a20326834c0c962f267bcb5708f8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:14.124190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e270a20326834c0c962f267bcb5708f8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:14.126250 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:14.359715 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e270a20326834c0c962f267bcb5708f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:14.367447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e270a20326834c0c962f267bcb5708f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:14.368123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e270a20326834c0c962f267bcb5708f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:14.368733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e270a20326834c0c962f267bcb5708f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:14.380881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b0fff440-98ca-40cd-bee6-a790afd6c0b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e270a20326834c0c962f267bcb5708f8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:14.464536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 973/1934] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:14 2022] DELETE /identity/v3/users/e270a20326834c0c962f267bcb5708f8 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:15.283731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:15.285271 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:15.298343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd334cf0-0a0a-4369-aad2-d7dd83ed9d69 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:15.370812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:15.371694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:15.371976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:15.372251 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] PATH_INFO: `/v3/projects/2cc069310621438f9eaa51f619ef52ef` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:15.373537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc069310621438f9eaa51f619ef52ef)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:15.375632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:15.537023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc069310621438f9eaa51f619ef52ef', 'request_id': 'req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:15.543219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc069310621438f9eaa51f619ef52ef', 'request_id': 'req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:15.636980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc069310621438f9eaa51f619ef52ef', 'request_id': 'req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:15.648085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] The token cache is being invalidate because project 2cc069310621438f9eaa51f619ef52ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:15.648368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:15.649200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ffe848e7-9427-4ad3-b80d-a69caa9dcfcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:15.651514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 962/1935] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:15 2022] DELETE /identity/v3/projects/2cc069310621438f9eaa51f619ef52ef => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:15.657563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18456f4b-6cdc-4438-b41c-7d4d5bbf7e03 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:15.658037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18456f4b-6cdc-4438-b41c-7d4d5bbf7e03 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:15.658209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18456f4b-6cdc-4438-b41c-7d4d5bbf7e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:15.682035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-18456f4b-6cdc-4438-b41c-7d4d5bbf7e03 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:15.683281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-18456f4b-6cdc-4438-b41c-7d4d5bbf7e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:15.769698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 974/1936] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:15.774654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30da15-b77d-4ba4-80fb-1f5cbdf3132d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:15.775970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30da15-b77d-4ba4-80fb-1f5cbdf3132d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:15.776275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30da15-b77d-4ba4-80fb-1f5cbdf3132d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:15.798022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-cc30da15-b77d-4ba4-80fb-1f5cbdf3132d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:15.799584 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-cc30da15-b77d-4ba4-80fb-1f5cbdf3132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:15.825649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 963/1937] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:15.826808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18456f4b-6cdc-4438-b41c-7d4d5bbf7e03 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:15.838376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18456f4b-6cdc-4438-b41c-7d4d5bbf7e03 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:15.843480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0c969-af09-4acf-9320-3e04e7cd1ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:15.847772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0c969-af09-4acf-9320-3e04e7cd1ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:15.847772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0c969-af09-4acf-9320-3e04e7cd1ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:15.847772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0c969-af09-4acf-9320-3e04e7cd1ec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:15.847772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0c969-af09-4acf-9320-3e04e7cd1ec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:15.847772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0c969-af09-4acf-9320-3e04e7cd1ec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:15.847772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0c969-af09-4acf-9320-3e04e7cd1ec8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:15.862613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 975/1938] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:10:15.867573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30da15-b77d-4ba4-80fb-1f5cbdf3132d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:15.880377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30da15-b77d-4ba4-80fb-1f5cbdf3132d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:15.887626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b762bff-37a9-4d76-8803-3d80c794e92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:15.888642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b762bff-37a9-4d76-8803-3d80c794e92d admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:15.889171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b762bff-37a9-4d76-8803-3d80c794e92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:15.894946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b762bff-37a9-4d76-8803-3d80c794e92d admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:15.894946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b762bff-37a9-4d76-8803-3d80c794e92d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:15.894946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b762bff-37a9-4d76-8803-3d80c794e92d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:16.014280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 964/1939] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:15 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:16.015017 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0c969-af09-4acf-9320-3e04e7cd1ec8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:16.029123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0c969-af09-4acf-9320-3e04e7cd1ec8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:16.074269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:16.110419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5a94d-831e-4e1f-9015-b7dd46373c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:16.113009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5a94d-831e-4e1f-9015-b7dd46373c65 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:16.113009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5a94d-831e-4e1f-9015-b7dd46373c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:16.113009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5a94d-831e-4e1f-9015-b7dd46373c65 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:16.113345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5a94d-831e-4e1f-9015-b7dd46373c65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:16.116070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5a94d-831e-4e1f-9015-b7dd46373c65 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:16.215465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 976/1940] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:16 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:16.216758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b762bff-37a9-4d76-8803-3d80c794e92d admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:16.227749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b762bff-37a9-4d76-8803-3d80c794e92d admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:16.232993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ed12b-df0e-47a2-aceb-35d051bd7315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:16.233787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ed12b-df0e-47a2-aceb-35d051bd7315 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:16.234326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ed12b-df0e-47a2-aceb-35d051bd7315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:16.234614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ed12b-df0e-47a2-aceb-35d051bd7315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:16.235385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ed12b-df0e-47a2-aceb-35d051bd7315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:16.237183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ed12b-df0e-47a2-aceb-35d051bd7315 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:16.248260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 965/1941] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:16.253553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5a94d-831e-4e1f-9015-b7dd46373c65 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:16.264017 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5a94d-831e-4e1f-9015-b7dd46373c65 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:16.268972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7ea22-5cb0-43db-bab5-3d7569aadbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:16.269640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ea22-5cb0-43db-bab5-3d7569aadbee admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:16.269930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ea22-5cb0-43db-bab5-3d7569aadbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:16.273008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ea22-5cb0-43db-bab5-3d7569aadbee admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:16.273008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7ea22-5cb0-43db-bab5-3d7569aadbee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:16.273249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7ea22-5cb0-43db-bab5-3d7569aadbee admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:16.283911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 977/1942] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:16.291133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ed12b-df0e-47a2-aceb-35d051bd7315 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:16.299469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ed12b-df0e-47a2-aceb-35d051bd7315 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:16.304533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e946da0-45a3-4b18-8ffb-bf18c323e3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:16.305492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e946da0-45a3-4b18-8ffb-bf18c323e3ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:16.305806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e946da0-45a3-4b18-8ffb-bf18c323e3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:16.306109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e946da0-45a3-4b18-8ffb-bf18c323e3ac admin admin] PATH_INFO: `/v3/projects/ccf40b38940c49bab9e8a3d8a16efb67/users/a0d59e7ada5d4c73b4ca181c8ee5f3da/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:16.318979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:16.325404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e946da0-45a3-4b18-8ffb-bf18c323e3ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf40b38940c49bab9e8a3d8a16efb67, user_id=a0d59e7ada5d4c73b4ca181c8ee5f3da, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:16.330451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e946da0-45a3-4b18-8ffb-bf18c323e3ac admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:16.413307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 966/1943] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:16 2022] PUT /identity/v3/projects/ccf40b38940c49bab9e8a3d8a16efb67/users/a0d59e7ada5d4c73b4ca181c8ee5f3da/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:16.424577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016a09-ccce-4274-bb7f-110e4330246d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:16.424918 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016a09-ccce-4274-bb7f-110e4330246d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:16.425240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34016a09-ccce-4274-bb7f-110e4330246d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:16.465485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-34016a09-ccce-4274-bb7f-110e4330246d None None] MFA Rules not processed for user `a0d59e7ada5d4c73b4ca181c8ee5f3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:16.470136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-34016a09-ccce-4274-bb7f-110e4330246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:16.568447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 978/1944] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:16.572969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e946da0-45a3-4b18-8ffb-bf18c323e3ac admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:16.572969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0e946da0-45a3-4b18-8ffb-bf18c323e3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:16.642312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e946da0-45a3-4b18-8ffb-bf18c323e3ac admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:16.647977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736628ed-5e5c-4cf9-b702-8fa2bf13a100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:16.648636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736628ed-5e5c-4cf9-b702-8fa2bf13a100 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:16.649121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736628ed-5e5c-4cf9-b702-8fa2bf13a100 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:16.649409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736628ed-5e5c-4cf9-b702-8fa2bf13a100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:16.661058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736628ed-5e5c-4cf9-b702-8fa2bf13a100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:16.663347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736628ed-5e5c-4cf9-b702-8fa2bf13a100 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:16.673895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 967/1945] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:16.692792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34016a09-ccce-4274-bb7f-110e4330246d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:16.771721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34016a09-ccce-4274-bb7f-110e4330246d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:16.776354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:16.777345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:16.777671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:16.777944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] PATH_INFO: `/v3/users/a0d59e7ada5d4c73b4ca181c8ee5f3da` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:16.784962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d59e7ada5d4c73b4ca181c8ee5f3da)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:16.786926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:17.013253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:17.020846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d59e7ada5d4c73b4ca181c8ee5f3da'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:17.030810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d59e7ada5d4c73b4ca181c8ee5f3da'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:17.032062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d59e7ada5d4c73b4ca181c8ee5f3da'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:17.035407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d59e7ada5d4c73b4ca181c8ee5f3da'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:17.046592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d59e7ada5d4c73b4ca181c8ee5f3da'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:17.125954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 979/1946] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:16 2022] DELETE /identity/v3/users/a0d59e7ada5d4c73b4ca181c8ee5f3da => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:17.130571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e92a9-8e41-4737-bc1f-0469027ffe0a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:17.131471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e92a9-8e41-4737-bc1f-0469027ffe0a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:17.131785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e92a9-8e41-4737-bc1f-0469027ffe0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:17.152068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-9d8e92a9-8e41-4737-bc1f-0469027ffe0a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:17.170941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9d8e92a9-8e41-4737-bc1f-0469027ffe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:17.174633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:17.179922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:17.244314 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 968/1947] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:17.259196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e92a9-8e41-4737-bc1f-0469027ffe0a None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:17.260750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9d8e92a9-8e41-4737-bc1f-0469027ffe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:17.286950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9a5f6-07b2-455a-837d-6e11518e2fe0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:17.294064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:17.295994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:17.296418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:17.296793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] PATH_INFO: `/v3/users/d89990bfd774417ab0eb52b91ec560d8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:17.298077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89990bfd774417ab0eb52b91ec560d8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:17.299999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:17.328287 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e92a9-8e41-4737-bc1f-0469027ffe0a None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:17.331959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b5860-f271-4a72-b858-0e23c97582d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:17.332604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b5860-f271-4a72-b858-0e23c97582d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:17.332925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b5860-f271-4a72-b858-0e23c97582d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:17.333219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b5860-f271-4a72-b858-0e23c97582d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:17.342944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b5860-f271-4a72-b858-0e23c97582d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:17.344371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b5860-f271-4a72-b858-0e23c97582d8 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:17.353766 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 969/1948] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:17.554705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89990bfd774417ab0eb52b91ec560d8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:17.561541 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89990bfd774417ab0eb52b91ec560d8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:17.562228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89990bfd774417ab0eb52b91ec560d8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:17.562869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89990bfd774417ab0eb52b91ec560d8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:17.575299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89990bfd774417ab0eb52b91ec560d8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:17.673276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 980/1949] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:17 2022] DELETE /identity/v3/users/d89990bfd774417ab0eb52b91ec560d8 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:17.844696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b5860-f271-4a72-b858-0e23c97582d8 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:17.846640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0e0b5860-f271-4a72-b858-0e23c97582d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:17.862146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b5860-f271-4a72-b858-0e23c97582d8 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:17.910054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:17.935840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:17.936906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:17.937195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:17.937479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] PATH_INFO: `/v3/projects/ccf40b38940c49bab9e8a3d8a16efb67` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:17.938702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf40b38940c49bab9e8a3d8a16efb67)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:17.940615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:17.994442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:17.994442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:18.005248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf8b5d-4e1c-4ab1-a974-26890b924e26 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.010413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.011276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.011557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.011833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] PATH_INFO: `/v3/projects/ba872aaf0fd84b1eac87e18659d222d5` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.013038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba872aaf0fd84b1eac87e18659d222d5)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.014957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.067288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf40b38940c49bab9e8a3d8a16efb67', 'request_id': 'req-1c0e696c-16ba-4348-81bc-7ad1b10959c4'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:18.074101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf40b38940c49bab9e8a3d8a16efb67', 'request_id': 'req-1c0e696c-16ba-4348-81bc-7ad1b10959c4'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:18.141164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:18.141888 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf40b38940c49bab9e8a3d8a16efb67', 'request_id': 'req-1c0e696c-16ba-4348-81bc-7ad1b10959c4'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:18.154336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba872aaf0fd84b1eac87e18659d222d5', 'request_id': 'req-a4db6556-b306-48bf-a09c-99430218fdd5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:18.154611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] The token cache is being invalidate because project ccf40b38940c49bab9e8a3d8a16efb67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:18.154908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:18.155541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1c0e696c-16ba-4348-81bc-7ad1b10959c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:18.157170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 970/1950] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:17 2022] DELETE /identity/v3/projects/ccf40b38940c49bab9e8a3d8a16efb67 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:18.159907 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba872aaf0fd84b1eac87e18659d222d5', 'request_id': 'req-a4db6556-b306-48bf-a09c-99430218fdd5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:18.161892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa7179-12d7-4668-9c42-5afee36eb95c None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.162892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa7179-12d7-4668-9c42-5afee36eb95c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.163234 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa7179-12d7-4668-9c42-5afee36eb95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.166731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba872aaf0fd84b1eac87e18659d222d5', 'request_id': 'req-a4db6556-b306-48bf-a09c-99430218fdd5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:18.182159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-40aa7179-12d7-4668-9c42-5afee36eb95c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:18.183595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-40aa7179-12d7-4668-9c42-5afee36eb95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:18.242571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 971/1951] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:18.250094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0373f07-f969-41d7-99da-f896e3191976 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.250375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0373f07-f969-41d7-99da-f896e3191976 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.250699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0373f07-f969-41d7-99da-f896e3191976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.255592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] The token cache is being invalidate because project ba872aaf0fd84b1eac87e18659d222d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:18.255887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:18.262539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a4db6556-b306-48bf-a09c-99430218fdd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:18.262539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 981/1952] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:17 2022] DELETE /identity/v3/projects/ba872aaf0fd84b1eac87e18659d222d5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:10:18.264968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa1ab-d523-4799-9524-d46b7fba99c0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.265362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa1ab-d523-4799-9524-d46b7fba99c0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.265717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa1ab-d523-4799-9524-d46b7fba99c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.275020 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-a0373f07-f969-41d7-99da-f896e3191976 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:18.276007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a0373f07-f969-41d7-99da-f896e3191976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:18.287727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ddcaa1ab-d523-4799-9524-d46b7fba99c0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:18.291870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 972/1953] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:18.292365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ddcaa1ab-d523-4799-9524-d46b7fba99c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:18.293535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0373f07-f969-41d7-99da-f896e3191976 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.297160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 982/1954] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:18.304830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0373f07-f969-41d7-99da-f896e3191976 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.312152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b3bfa-3b81-42aa-9833-35c54a0e8902 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.312152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b3bfa-3b81-42aa-9833-35c54a0e8902 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.312152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b3bfa-3b81-42aa-9833-35c54a0e8902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.312655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6667ce-601d-4d25-b765-10cf5aae7db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.320560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6667ce-601d-4d25-b765-10cf5aae7db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.320906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6667ce-601d-4d25-b765-10cf5aae7db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.321179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6667ce-601d-4d25-b765-10cf5aae7db3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.321947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6667ce-601d-4d25-b765-10cf5aae7db3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:18.322273 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6667ce-601d-4d25-b765-10cf5aae7db3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.323912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6667ce-601d-4d25-b765-10cf5aae7db3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.328849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-218b3bfa-3b81-42aa-9833-35c54a0e8902 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:18.330419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-218b3bfa-3b81-42aa-9833-35c54a0e8902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:18.336413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 973/1955] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:10:18.340740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6667ce-601d-4d25-b765-10cf5aae7db3 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.343554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 983/1956] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:18.348136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b3bfa-3b81-42aa-9833-35c54a0e8902 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.354926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6667ce-601d-4d25-b765-10cf5aae7db3 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.359938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524a58-51df-4948-9127-78e464e64c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.360601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524a58-51df-4948-9127-78e464e64c1e admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.360941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524a58-51df-4948-9127-78e464e64c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.361206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524a58-51df-4948-9127-78e464e64c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.361896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524a58-51df-4948-9127-78e464e64c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.363383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b3bfa-3b81-42aa-9833-35c54a0e8902 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.363815 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524a58-51df-4948-9127-78e464e64c1e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.370044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917daa30-7602-4a92-b787-10893a550319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.370803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917daa30-7602-4a92-b787-10893a550319 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.371139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917daa30-7602-4a92-b787-10893a550319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.371471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917daa30-7602-4a92-b787-10893a550319 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.374593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917daa30-7602-4a92-b787-10893a550319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:18.374593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917daa30-7602-4a92-b787-10893a550319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.374807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917daa30-7602-4a92-b787-10893a550319 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.388364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 984/1957] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:10:18.392487 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917daa30-7602-4a92-b787-10893a550319 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.402961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917daa30-7602-4a92-b787-10893a550319 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.407648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fcc38-67e3-40e0-bf0b-52b286aa0f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.408351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fcc38-67e3-40e0-bf0b-52b286aa0f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.408696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fcc38-67e3-40e0-bf0b-52b286aa0f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.409008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fcc38-67e3-40e0-bf0b-52b286aa0f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.409844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fcc38-67e3-40e0-bf0b-52b286aa0f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.411757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fcc38-67e3-40e0-bf0b-52b286aa0f62 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.481380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 974/1958] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:18 2022] POST /identity/v3/projects => generated 369 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:18.486237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524a58-51df-4948-9127-78e464e64c1e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.514436 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:18.532001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 985/1959] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:18 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:18.536156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fcc38-67e3-40e0-bf0b-52b286aa0f62 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.547643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524a58-51df-4948-9127-78e464e64c1e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.600145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fcc38-67e3-40e0-bf0b-52b286aa0f62 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.603852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7727164-0d2b-4a84-aff8-0888fd488435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.604525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7727164-0d2b-4a84-aff8-0888fd488435 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.604818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8a739-26f9-47fa-a4aa-b6220b92f3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.604938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7727164-0d2b-4a84-aff8-0888fd488435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.605178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7727164-0d2b-4a84-aff8-0888fd488435 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.605540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8a739-26f9-47fa-a4aa-b6220b92f3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.605832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8a739-26f9-47fa-a4aa-b6220b92f3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.605930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7727164-0d2b-4a84-aff8-0888fd488435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.606147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8a739-26f9-47fa-a4aa-b6220b92f3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.606834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8a739-26f9-47fa-a4aa-b6220b92f3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.607759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7727164-0d2b-4a84-aff8-0888fd488435 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.608752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8a739-26f9-47fa-a4aa-b6220b92f3a8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.713884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 975/1960] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:18 2022] POST /identity/v3/users => generated 285 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:18.717880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7727164-0d2b-4a84-aff8-0888fd488435 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.728262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7727164-0d2b-4a84-aff8-0888fd488435 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.729324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 986/1961] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:18 2022] POST /identity/v3/users => generated 282 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:18.730218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8a739-26f9-47fa-a4aa-b6220b92f3a8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.732942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ff2e5-89e5-4b81-9db4-8ba2cc30f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.733592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ff2e5-89e5-4b81-9db4-8ba2cc30f9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.733908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ff2e5-89e5-4b81-9db4-8ba2cc30f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.734466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ff2e5-89e5-4b81-9db4-8ba2cc30f9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.740616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ff2e5-89e5-4b81-9db4-8ba2cc30f9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.740616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ff2e5-89e5-4b81-9db4-8ba2cc30f9b8 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.740901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8a739-26f9-47fa-a4aa-b6220b92f3a8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.744737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06b786-6e2a-4b37-b095-34c23ac82b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.745417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06b786-6e2a-4b37-b095-34c23ac82b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.745702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06b786-6e2a-4b37-b095-34c23ac82b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.746036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06b786-6e2a-4b37-b095-34c23ac82b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.746766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06b786-6e2a-4b37-b095-34c23ac82b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.746898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 976/1962] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:10:18.748356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06b786-6e2a-4b37-b095-34c23ac82b4b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.750558 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ff2e5-89e5-4b81-9db4-8ba2cc30f9b8 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.760874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 987/1963] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:18.761037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ff2e5-89e5-4b81-9db4-8ba2cc30f9b8 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.765609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0aac9-afe8-4158-8509-5e0135b94b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.766104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0aac9-afe8-4158-8509-5e0135b94b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.766395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0aac9-afe8-4158-8509-5e0135b94b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.766547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06b786-6e2a-4b37-b095-34c23ac82b4b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.766674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0aac9-afe8-4158-8509-5e0135b94b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.767385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0aac9-afe8-4158-8509-5e0135b94b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.768990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0aac9-afe8-4158-8509-5e0135b94b29 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.776711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06b786-6e2a-4b37-b095-34c23ac82b4b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.780900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 977/1964] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:10:18.782956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be6a56-5053-4fbd-89ba-6e6b747d0ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.782956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be6a56-5053-4fbd-89ba-6e6b747d0ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.782956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be6a56-5053-4fbd-89ba-6e6b747d0ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.782956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be6a56-5053-4fbd-89ba-6e6b747d0ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.783586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be6a56-5053-4fbd-89ba-6e6b747d0ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.784012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0aac9-afe8-4158-8509-5e0135b94b29 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.785267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be6a56-5053-4fbd-89ba-6e6b747d0ef5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.795532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0aac9-afe8-4158-8509-5e0135b94b29 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.796383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 988/1965] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:10:18.801545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be6a56-5053-4fbd-89ba-6e6b747d0ef5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:18.801943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60668e8a-91e7-4149-b62b-7d780ca4ebf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.802944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668e8a-91e7-4149-b62b-7d780ca4ebf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.803225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668e8a-91e7-4149-b62b-7d780ca4ebf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.803498 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668e8a-91e7-4149-b62b-7d780ca4ebf9 admin admin] PATH_INFO: `/v3/projects/532ea5c6fc364dc79591e551e2d3b8eb/users/8311a61e39b64f00b9235973f7414cfe/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.815723 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be6a56-5053-4fbd-89ba-6e6b747d0ef5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:18.819564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60668e8a-91e7-4149-b62b-7d780ca4ebf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532ea5c6fc364dc79591e551e2d3b8eb, user_id=8311a61e39b64f00b9235973f7414cfe, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.821594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61475053-efd5-45ce-b328-1a2a2de13991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:18.822462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60668e8a-91e7-4149-b62b-7d780ca4ebf9 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.822575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61475053-efd5-45ce-b328-1a2a2de13991 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.822844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61475053-efd5-45ce-b328-1a2a2de13991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.823123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61475053-efd5-45ce-b328-1a2a2de13991 admin admin] PATH_INFO: `/v3/projects/4b693711a116429abddde9db8816d7bc/users/af45df9a59434a338d36623d6282cf85/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.839425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61475053-efd5-45ce-b328-1a2a2de13991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b693711a116429abddde9db8816d7bc, user_id=af45df9a59434a338d36623d6282cf85, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:18.842292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61475053-efd5-45ce-b328-1a2a2de13991 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:18.941554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 978/1966] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:18 2022] PUT /identity/v3/projects/532ea5c6fc364dc79591e551e2d3b8eb/users/8311a61e39b64f00b9235973f7414cfe/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:18.954453 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 989/1967] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:18 2022] PUT /identity/v3/projects/4b693711a116429abddde9db8816d7bc/users/af45df9a59434a338d36623d6282cf85/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:10:18.955674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1210663-fdc6-407f-ae6d-9d6beccb4efd None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.955976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1210663-fdc6-407f-ae6d-9d6beccb4efd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.956259 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1210663-fdc6-407f-ae6d-9d6beccb4efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.964679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f474a4b-3668-4093-b9e6-cf20cd6c45ff None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:18.964988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f474a4b-3668-4093-b9e6-cf20cd6c45ff None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:18.965191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f474a4b-3668-4093-b9e6-cf20cd6c45ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:18.993879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e1210663-fdc6-407f-ae6d-9d6beccb4efd None None] MFA Rules not processed for user `8311a61e39b64f00b9235973f7414cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:18.995309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e1210663-fdc6-407f-ae6d-9d6beccb4efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:19.000908 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:19.002991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3f474a4b-3668-4093-b9e6-cf20cd6c45ff None None] MFA Rules not processed for user `af45df9a59434a338d36623d6282cf85`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:19.004220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3f474a4b-3668-4093-b9e6-cf20cd6c45ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:19.061882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 979/1968] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:19.073328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 990/1969] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:19.075553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1210663-fdc6-407f-ae6d-9d6beccb4efd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:19.076959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e1210663-fdc6-407f-ae6d-9d6beccb4efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:19.077427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f474a4b-3668-4093-b9e6-cf20cd6c45ff None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:19.089385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:19.140465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1210663-fdc6-407f-ae6d-9d6beccb4efd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:19.144382 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0edf1d-8037-4d08-918a-6e0832ae69c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b712bd7d8d4930a3bfc99581be8cb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:19.145065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0edf1d-8037-4d08-918a-6e0832ae69c9 service glance] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:19.145601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0edf1d-8037-4d08-918a-6e0832ae69c9 service glance] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:19.145884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0edf1d-8037-4d08-918a-6e0832ae69c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:19.151708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f474a4b-3668-4093-b9e6-cf20cd6c45ff None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:19.156174 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0edf1d-8037-4d08-918a-6e0832ae69c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:19.157325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed61301-ab2e-4f83-8201-ea5e177602eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:19.157651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0edf1d-8037-4d08-918a-6e0832ae69c9 service glance] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:19.159313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61301-ab2e-4f83-8201-ea5e177602eb admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:19.159313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61301-ab2e-4f83-8201-ea5e177602eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:19.159313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61301-ab2e-4f83-8201-ea5e177602eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:19.159568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed61301-ab2e-4f83-8201-ea5e177602eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:19.161642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed61301-ab2e-4f83-8201-ea5e177602eb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:19.169195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 980/1970] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:19 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:10:19.260207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 991/1971] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:19 2022] POST /identity/v3/projects => generated 363 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:19.264915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0edf1d-8037-4d08-918a-6e0832ae69c9 service glance] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:19.281480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0edf1d-8037-4d08-918a-6e0832ae69c9 service glance] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:19.355515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad67a35-1f12-403d-a481-02c557fbc722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:19.356245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad67a35-1f12-403d-a481-02c557fbc722 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:19.356485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad67a35-1f12-403d-a481-02c557fbc722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:19.356774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad67a35-1f12-403d-a481-02c557fbc722 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:19.357537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad67a35-1f12-403d-a481-02c557fbc722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:19.359763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad67a35-1f12-403d-a481-02c557fbc722 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:19.456797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 981/1972] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:19 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:19.458738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed61301-ab2e-4f83-8201-ea5e177602eb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:19.470962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed61301-ab2e-4f83-8201-ea5e177602eb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:19.477040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eb7ff-c57c-4253-b2cd-7cf77b4da572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:19.477883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eb7ff-c57c-4253-b2cd-7cf77b4da572 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:19.478905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eb7ff-c57c-4253-b2cd-7cf77b4da572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:19.479251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eb7ff-c57c-4253-b2cd-7cf77b4da572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:19.480029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95eb7ff-c57c-4253-b2cd-7cf77b4da572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:19.486506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95eb7ff-c57c-4253-b2cd-7cf77b4da572 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:19.496126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 992/1973] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:10:19.500603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad67a35-1f12-403d-a481-02c557fbc722 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:19.512379 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad67a35-1f12-403d-a481-02c557fbc722 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:19.517092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de41b7-2e4a-4294-816e-a3d33ed96ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:19.520965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de41b7-2e4a-4294-816e-a3d33ed96ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:19.520965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de41b7-2e4a-4294-816e-a3d33ed96ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:19.520965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de41b7-2e4a-4294-816e-a3d33ed96ebe admin admin] PATH_INFO: `/v3/projects/a22130e5eb144633992405b5cd25c917/users/56505140076943c191c5ce2480533a1d/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:19.534476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de41b7-2e4a-4294-816e-a3d33ed96ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22130e5eb144633992405b5cd25c917, user_id=56505140076943c191c5ce2480533a1d, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:19.537240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de41b7-2e4a-4294-816e-a3d33ed96ebe admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:19.616132 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 982/1974] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:19 2022] PUT /identity/v3/projects/a22130e5eb144633992405b5cd25c917/users/56505140076943c191c5ce2480533a1d/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:19.619930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eb7ff-c57c-4253-b2cd-7cf77b4da572 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:19.630074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eb7ff-c57c-4253-b2cd-7cf77b4da572 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:19.686269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f5335-dc95-4129-a0f5-0f02e1e7c509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:19.686919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f5335-dc95-4129-a0f5-0f02e1e7c509 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:19.687201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f5335-dc95-4129-a0f5-0f02e1e7c509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:19.687548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f5335-dc95-4129-a0f5-0f02e1e7c509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:19.688207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f5335-dc95-4129-a0f5-0f02e1e7c509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:19.689815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f5335-dc95-4129-a0f5-0f02e1e7c509 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:19.700272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 993/1975] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:19.704451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de41b7-2e4a-4294-816e-a3d33ed96ebe admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:19.715565 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de41b7-2e4a-4294-816e-a3d33ed96ebe admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:19.720329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b7f1a-f11b-4279-9090-371c700b8015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:19.721852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7f1a-f11b-4279-9090-371c700b8015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:19.721852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7f1a-f11b-4279-9090-371c700b8015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:19.721852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7f1a-f11b-4279-9090-371c700b8015 admin admin] PATH_INFO: `/v3/projects/a22130e5eb144633992405b5cd25c917/users/56505140076943c191c5ce2480533a1d/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:19.723849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b7f1a-f11b-4279-9090-371c700b8015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22130e5eb144633992405b5cd25c917, user_id=56505140076943c191c5ce2480533a1d, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:19.726704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b7f1a-f11b-4279-9090-371c700b8015 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:19.737229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 983/1976] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:19 2022] PUT /identity/v3/projects/a22130e5eb144633992405b5cd25c917/users/56505140076943c191c5ce2480533a1d/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:19.747586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0be266-f872-4590-8094-5a6a9bff6e5e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:19.750622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0be266-f872-4590-8094-5a6a9bff6e5e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:19.750622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0be266-f872-4590-8094-5a6a9bff6e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:19.773438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:19.776408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2b0be266-f872-4590-8094-5a6a9bff6e5e None None] MFA Rules not processed for user `56505140076943c191c5ce2480533a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:19.778810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2b0be266-f872-4590-8094-5a6a9bff6e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:19.846890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 994/1977] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:19 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:19.858230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b7f1a-f11b-4279-9090-371c700b8015 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:19.860151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d33b7f1a-f11b-4279-9090-371c700b8015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:19.917573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b7f1a-f11b-4279-9090-371c700b8015 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:19.921190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cb50a-6c6c-4b68-b2d9-a830ee75a80b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:19.921827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb50a-6c6c-4b68-b2d9-a830ee75a80b service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:19.922114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb50a-6c6c-4b68-b2d9-a830ee75a80b service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:19.922389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb50a-6c6c-4b68-b2d9-a830ee75a80b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:19.932743 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cb50a-6c6c-4b68-b2d9-a830ee75a80b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:19.934188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cb50a-6c6c-4b68-b2d9-a830ee75a80b service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:19.944497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 984/1978] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:19 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:20.176256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0be266-f872-4590-8094-5a6a9bff6e5e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:20.186638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0be266-f872-4590-8094-5a6a9bff6e5e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:20.190311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c7d5d-1849-4291-af64-171ea4de7059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:20.190951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c7d5d-1849-4291-af64-171ea4de7059 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:20.191234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c7d5d-1849-4291-af64-171ea4de7059 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:20.191506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c7d5d-1849-4291-af64-171ea4de7059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:20.249271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c7d5d-1849-4291-af64-171ea4de7059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:20.250711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c7d5d-1849-4291-af64-171ea4de7059 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:20.259566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 995/1979] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:20 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:10:22.951729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cb50a-6c6c-4b68-b2d9-a830ee75a80b service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:22.953208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f83cb50a-6c6c-4b68-b2d9-a830ee75a80b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:22.959703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:23.020300 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cb50a-6c6c-4b68-b2d9-a830ee75a80b service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:23.025447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:23.026419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:23.026791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:23.027118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] PATH_INFO: `/v3/users/03f47d055a894b9bb8b423ddfc3efeb0` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:23.028404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f47d055a894b9bb8b423ddfc3efeb0)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:23.030579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:23.268088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f47d055a894b9bb8b423ddfc3efeb0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:23.277983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f47d055a894b9bb8b423ddfc3efeb0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:23.278683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f47d055a894b9bb8b423ddfc3efeb0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:23.298469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f47d055a894b9bb8b423ddfc3efeb0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:23.299721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f47d055a894b9bb8b423ddfc3efeb0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:23.387994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 985/1980] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:22 2022] DELETE /identity/v3/users/03f47d055a894b9bb8b423ddfc3efeb0 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:23.392594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29fd01-b0e5-4b20-be5d-5edf3fb354d8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:23.393090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29fd01-b0e5-4b20-be5d-5edf3fb354d8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:23.393305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29fd01-b0e5-4b20-be5d-5edf3fb354d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:23.411385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-bc29fd01-b0e5-4b20-be5d-5edf3fb354d8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:23.413205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bc29fd01-b0e5-4b20-be5d-5edf3fb354d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:23.487262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 996/1981] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:23.496668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:23.499205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:23.574319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb49c74-031e-4329-987b-264a67b9db27 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:23.578085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0044c-ddde-4739-ae72-76497c36b7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:23.579811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0044c-ddde-4739-ae72-76497c36b7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:23.579811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0044c-ddde-4739-ae72-76497c36b7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:23.579811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0044c-ddde-4739-ae72-76497c36b7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:23.589604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0044c-ddde-4739-ae72-76497c36b7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:23.590542 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0044c-ddde-4739-ae72-76497c36b7d3 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:23.600602 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 986/1982] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:24.044667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29fd01-b0e5-4b20-be5d-5edf3fb354d8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:24.046922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bc29fd01-b0e5-4b20-be5d-5edf3fb354d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:24.091575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29fd01-b0e5-4b20-be5d-5edf3fb354d8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:24.096808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:24.101061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:24.101061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:24.101061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] PATH_INFO: `/v3/projects/fab222ce3b0041329c6971676dedd756` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:24.102620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab222ce3b0041329c6971676dedd756)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:24.104714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:24.208584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:24.221241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab222ce3b0041329c6971676dedd756', 'request_id': 'req-82336da2-934f-4581-8fe2-007c8db65b55'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:24.228217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab222ce3b0041329c6971676dedd756', 'request_id': 'req-82336da2-934f-4581-8fe2-007c8db65b55'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:24.235910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab222ce3b0041329c6971676dedd756', 'request_id': 'req-82336da2-934f-4581-8fe2-007c8db65b55'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:24.362805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] The token cache is being invalidate because project fab222ce3b0041329c6971676dedd756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:24.363115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:24.363757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:24.368739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 997/1983] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:24 2022] DELETE /identity/v3/projects/fab222ce3b0041329c6971676dedd756 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:24.370240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0044c-ddde-4739-ae72-76497c36b7d3 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:24.380066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0044c-ddde-4739-ae72-76497c36b7d3 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:24.406734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:24.439825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:24.440682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:24.440991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:24.441266 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] PATH_INFO: `/v3/users/e1446cf4141e4fe19a5bdb3c65f2dee7` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:24.442434 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1446cf4141e4fe19a5bdb3c65f2dee7)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:24.445085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:24.804582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1446cf4141e4fe19a5bdb3c65f2dee7'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:24.811454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1446cf4141e4fe19a5bdb3c65f2dee7'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:24.812131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1446cf4141e4fe19a5bdb3c65f2dee7'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:24.823574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1446cf4141e4fe19a5bdb3c65f2dee7'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:24.824222 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b178e2ff-3bdd-4c69-82ef-55e66f587a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1446cf4141e4fe19a5bdb3c65f2dee7'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:24.882518 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 987/1984] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:24 2022] DELETE /identity/v3/users/e1446cf4141e4fe19a5bdb3c65f2dee7 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:25.381595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:25.383170 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:25.397060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82336da2-934f-4581-8fe2-007c8db65b55 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:25.456828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:25.457719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:25.458013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:25.458272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] PATH_INFO: `/v3/projects/efc4ae54a9f748bb971a3488b560d7ae` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:25.459479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc4ae54a9f748bb971a3488b560d7ae)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:25.461457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:25.603332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc4ae54a9f748bb971a3488b560d7ae', 'request_id': 'req-cb25b910-d911-49bc-a018-946ff77062d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:25.615861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc4ae54a9f748bb971a3488b560d7ae', 'request_id': 'req-cb25b910-d911-49bc-a018-946ff77062d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:25.622476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc4ae54a9f748bb971a3488b560d7ae', 'request_id': 'req-cb25b910-d911-49bc-a018-946ff77062d9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:25.729949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] The token cache is being invalidate because project efc4ae54a9f748bb971a3488b560d7ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:25.730239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:25.730848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-cb25b910-d911-49bc-a018-946ff77062d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:25.732426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 998/1985] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:25 2022] DELETE /identity/v3/projects/efc4ae54a9f748bb971a3488b560d7ae => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:25.737593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877a787-d60e-4518-be3d-7608194126cc None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:25.738137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877a787-d60e-4518-be3d-7608194126cc None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:25.738137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877a787-d60e-4518-be3d-7608194126cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:25.757668 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1877a787-d60e-4518-be3d-7608194126cc None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:25.759942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1877a787-d60e-4518-be3d-7608194126cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:25.821377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 988/1986] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:25.829428 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f303aa9-5655-43a3-a818-5f409755d6a8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:25.829814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f303aa9-5655-43a3-a818-5f409755d6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:25.830208 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f303aa9-5655-43a3-a818-5f409755d6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:25.851956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4f303aa9-5655-43a3-a818-5f409755d6a8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:25.853293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4f303aa9-5655-43a3-a818-5f409755d6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:25.863304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 999/1987] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:25.868138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877a787-d60e-4518-be3d-7608194126cc None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:25.877689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877a787-d60e-4518-be3d-7608194126cc None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:25.882514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc52ed-1627-4533-ac7d-91721776189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:25.883177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc52ed-1627-4533-ac7d-91721776189a admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:25.883470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc52ed-1627-4533-ac7d-91721776189a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:25.883746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc52ed-1627-4533-ac7d-91721776189a admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:25.884475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc52ed-1627-4533-ac7d-91721776189a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:25.884817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc52ed-1627-4533-ac7d-91721776189a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:25.886460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc52ed-1627-4533-ac7d-91721776189a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:25.898983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 989/1988] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:10:25.903322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f303aa9-5655-43a3-a818-5f409755d6a8 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:25.913393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f303aa9-5655-43a3-a818-5f409755d6a8 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:25.918089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1655d2-d781-4ff4-ae01-6b936e6630e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:25.918938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1655d2-d781-4ff4-ae01-6b936e6630e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:25.919258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1655d2-d781-4ff4-ae01-6b936e6630e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:25.919540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1655d2-d781-4ff4-ae01-6b936e6630e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:25.920293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1655d2-d781-4ff4-ae01-6b936e6630e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:25.922704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1655d2-d781-4ff4-ae01-6b936e6630e9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:26.024962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1000/1989] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:25 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:26.029255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc52ed-1627-4533-ac7d-91721776189a admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:26.039992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc52ed-1627-4533-ac7d-91721776189a admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:26.069912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:26.096435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a509f29-2999-4829-acee-e6cf1e164035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:26.097136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a509f29-2999-4829-acee-e6cf1e164035 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:26.097479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a509f29-2999-4829-acee-e6cf1e164035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:26.097763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a509f29-2999-4829-acee-e6cf1e164035 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:26.098664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a509f29-2999-4829-acee-e6cf1e164035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:26.101880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a509f29-2999-4829-acee-e6cf1e164035 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:26.185468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 990/1990] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:26 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:26.189342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1655d2-d781-4ff4-ae01-6b936e6630e9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:26.201923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1655d2-d781-4ff4-ae01-6b936e6630e9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:26.209084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b6167-ceb0-4e49-93ec-5dcac2707093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:26.209750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b6167-ceb0-4e49-93ec-5dcac2707093 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:26.210079 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b6167-ceb0-4e49-93ec-5dcac2707093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:26.215016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b6167-ceb0-4e49-93ec-5dcac2707093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:26.215016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b6167-ceb0-4e49-93ec-5dcac2707093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:26.215016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b6167-ceb0-4e49-93ec-5dcac2707093 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:26.223880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1001/1991] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:26.228341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a509f29-2999-4829-acee-e6cf1e164035 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:26.238590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a509f29-2999-4829-acee-e6cf1e164035 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:26.243316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903d263-206d-4cec-8a59-ec70a366d339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:26.243992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903d263-206d-4cec-8a59-ec70a366d339 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:26.244358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903d263-206d-4cec-8a59-ec70a366d339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:26.244657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903d263-206d-4cec-8a59-ec70a366d339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:26.245450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903d263-206d-4cec-8a59-ec70a366d339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:26.247734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903d263-206d-4cec-8a59-ec70a366d339 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:26.259684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 991/1992] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:26.263780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b6167-ceb0-4e49-93ec-5dcac2707093 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:26.274956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b6167-ceb0-4e49-93ec-5dcac2707093 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:26.279517 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba3067-6fc0-4cd7-8bfc-6141b706888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:26.280493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba3067-6fc0-4cd7-8bfc-6141b706888c admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:26.280851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba3067-6fc0-4cd7-8bfc-6141b706888c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:26.281187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba3067-6fc0-4cd7-8bfc-6141b706888c admin admin] PATH_INFO: `/v3/projects/39a3d9d44e7044d1a3d6fb780e0f79db/users/3875acc8fcc84853a25c89bf934eae2c/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:26.290485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:26.299904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba3067-6fc0-4cd7-8bfc-6141b706888c admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a3d9d44e7044d1a3d6fb780e0f79db, user_id=3875acc8fcc84853a25c89bf934eae2c, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:26.302686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba3067-6fc0-4cd7-8bfc-6141b706888c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:26.386445 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1002/1993] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:26 2022] PUT /identity/v3/projects/39a3d9d44e7044d1a3d6fb780e0f79db/users/3875acc8fcc84853a25c89bf934eae2c/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:26.398396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc5ac2-c57e-454a-b522-052903f9b4dd None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:26.398687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc5ac2-c57e-454a-b522-052903f9b4dd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:26.398962 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc5ac2-c57e-454a-b522-052903f9b4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:26.426315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-54dc5ac2-c57e-454a-b522-052903f9b4dd None None] MFA Rules not processed for user `3875acc8fcc84853a25c89bf934eae2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:26.427748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-54dc5ac2-c57e-454a-b522-052903f9b4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:26.492047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 992/1994] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:26.496418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba3067-6fc0-4cd7-8bfc-6141b706888c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:26.557656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba3067-6fc0-4cd7-8bfc-6141b706888c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:26.562400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8ea62-b428-4b29-a141-7a1c469d5875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:26.563062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ea62-b428-4b29-a141-7a1c469d5875 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:26.563366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ea62-b428-4b29-a141-7a1c469d5875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:26.563630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ea62-b428-4b29-a141-7a1c469d5875 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:26.564326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8ea62-b428-4b29-a141-7a1c469d5875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:26.566255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8ea62-b428-4b29-a141-7a1c469d5875 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:26.678677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1003/1995] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:26 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:26.682381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc5ac2-c57e-454a-b522-052903f9b4dd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:26.693551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc5ac2-c57e-454a-b522-052903f9b4dd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:26.751388 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67bbbb-9211-4463-bdb3-915be15b3833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:26.752071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67bbbb-9211-4463-bdb3-915be15b3833 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:26.752359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67bbbb-9211-4463-bdb3-915be15b3833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:26.752636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67bbbb-9211-4463-bdb3-915be15b3833 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:26.753378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67bbbb-9211-4463-bdb3-915be15b3833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:26.755334 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67bbbb-9211-4463-bdb3-915be15b3833 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:26.875809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 993/1996] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:26 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:26.876846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8ea62-b428-4b29-a141-7a1c469d5875 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:26.889572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8ea62-b428-4b29-a141-7a1c469d5875 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:26.894503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4865c-5ce5-48ce-83a1-f6c247c4aaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:26.895182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4865c-5ce5-48ce-83a1-f6c247c4aaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:26.895467 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4865c-5ce5-48ce-83a1-f6c247c4aaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:26.895750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4865c-5ce5-48ce-83a1-f6c247c4aaf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:26.896549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4865c-5ce5-48ce-83a1-f6c247c4aaf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:26.898194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4865c-5ce5-48ce-83a1-f6c247c4aaf7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:26.915047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1004/1997] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:26.918068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67bbbb-9211-4463-bdb3-915be15b3833 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:26.930979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67bbbb-9211-4463-bdb3-915be15b3833 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:26.935608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045cb5c-5e7c-4464-aea4-8af7e0d6f5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:26.936563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045cb5c-5e7c-4464-aea4-8af7e0d6f5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:26.936942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045cb5c-5e7c-4464-aea4-8af7e0d6f5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:26.937281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045cb5c-5e7c-4464-aea4-8af7e0d6f5e4 admin admin] PATH_INFO: `/v3/projects/a4e32fc3af1e48d687ec84179716146a/users/27fc61eaf82647fbbecc138042bf795a/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:26.954646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045cb5c-5e7c-4464-aea4-8af7e0d6f5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e32fc3af1e48d687ec84179716146a, user_id=27fc61eaf82647fbbecc138042bf795a, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:26.957398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045cb5c-5e7c-4464-aea4-8af7e0d6f5e4 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:27.026097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 994/1998] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:26 2022] PUT /identity/v3/projects/a4e32fc3af1e48d687ec84179716146a/users/27fc61eaf82647fbbecc138042bf795a/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:27.030201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4865c-5ce5-48ce-83a1-f6c247c4aaf7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:27.041237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4865c-5ce5-48ce-83a1-f6c247c4aaf7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:27.107412 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef54146-91fe-43d0-b6ab-88afe1ae7975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:27.108088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef54146-91fe-43d0-b6ab-88afe1ae7975 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:27.108374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef54146-91fe-43d0-b6ab-88afe1ae7975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:27.108648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef54146-91fe-43d0-b6ab-88afe1ae7975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:27.109695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef54146-91fe-43d0-b6ab-88afe1ae7975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:27.111285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef54146-91fe-43d0-b6ab-88afe1ae7975 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:27.122352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1005/1999] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:27.126795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045cb5c-5e7c-4464-aea4-8af7e0d6f5e4 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:27.137123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045cb5c-5e7c-4464-aea4-8af7e0d6f5e4 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:27.141613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db8e59-fe8f-4c84-96b1-86c0b33dd52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:27.142455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db8e59-fe8f-4c84-96b1-86c0b33dd52e admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:27.142737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db8e59-fe8f-4c84-96b1-86c0b33dd52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:27.143013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db8e59-fe8f-4c84-96b1-86c0b33dd52e admin admin] PATH_INFO: `/v3/projects/a4e32fc3af1e48d687ec84179716146a/users/27fc61eaf82647fbbecc138042bf795a/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:27.144988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db8e59-fe8f-4c84-96b1-86c0b33dd52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e32fc3af1e48d687ec84179716146a, user_id=27fc61eaf82647fbbecc138042bf795a, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:27.147674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db8e59-fe8f-4c84-96b1-86c0b33dd52e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:27.149874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:27.158895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 995/2000] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:27 2022] PUT /identity/v3/projects/a4e32fc3af1e48d687ec84179716146a/users/27fc61eaf82647fbbecc138042bf795a/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:27.172006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a8d92-0249-41af-9d06-a17276d943ec None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:27.172301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a8d92-0249-41af-9d06-a17276d943ec None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:27.172641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a8d92-0249-41af-9d06-a17276d943ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:27.199689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1f7a8d92-0249-41af-9d06-a17276d943ec None None] MFA Rules not processed for user `27fc61eaf82647fbbecc138042bf795a`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:27.200953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1f7a8d92-0249-41af-9d06-a17276d943ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:27.248864 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:27.274170 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1006/2001] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:27.285644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db8e59-fe8f-4c84-96b1-86c0b33dd52e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:27.288036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-99db8e59-fe8f-4c84-96b1-86c0b33dd52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:27.346614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db8e59-fe8f-4c84-96b1-86c0b33dd52e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:27.350787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fde997-dd2d-4953-8585-a9cf2344ab41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:27.351442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fde997-dd2d-4953-8585-a9cf2344ab41 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:27.351726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fde997-dd2d-4953-8585-a9cf2344ab41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:27.352000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fde997-dd2d-4953-8585-a9cf2344ab41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:27.361282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fde997-dd2d-4953-8585-a9cf2344ab41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:27.363202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fde997-dd2d-4953-8585-a9cf2344ab41 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:27.375550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 996/2002] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:27.546932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a8d92-0249-41af-9d06-a17276d943ec None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:27.556277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a8d92-0249-41af-9d06-a17276d943ec None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:27.559837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894c21a-34b6-4066-8398-7b8512907e05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:27.560456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894c21a-34b6-4066-8398-7b8512907e05 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:27.560740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894c21a-34b6-4066-8398-7b8512907e05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:27.561047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894c21a-34b6-4066-8398-7b8512907e05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:27.624620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894c21a-34b6-4066-8398-7b8512907e05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:27.626695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894c21a-34b6-4066-8398-7b8512907e05 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:27.638237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1007/2003] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:29.281107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fde997-dd2d-4953-8585-a9cf2344ab41 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:29.282498 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-20fde997-dd2d-4953-8585-a9cf2344ab41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:29.383563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fde997-dd2d-4953-8585-a9cf2344ab41 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:29.388302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:29.389279 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:29.389562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:29.389838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] PATH_INFO: `/v3/users/536cba541cc04f4abdad741dd391d9ce` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:29.391055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=536cba541cc04f4abdad741dd391d9ce)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:29.393034 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:29.764898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '536cba541cc04f4abdad741dd391d9ce'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:29.772642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '536cba541cc04f4abdad741dd391d9ce'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:29.773486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '536cba541cc04f4abdad741dd391d9ce'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:29.797859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '536cba541cc04f4abdad741dd391d9ce'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:29.798590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '536cba541cc04f4abdad741dd391d9ce'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:29.890310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 997/2004] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:29 2022] DELETE /identity/v3/users/536cba541cc04f4abdad741dd391d9ce => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:29.894719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c40e2-b76b-452f-9863-5a7355a5c21b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:29.895198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c40e2-b76b-452f-9863-5a7355a5c21b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:29.895296 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c40e2-b76b-452f-9863-5a7355a5c21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:29.913549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f04c40e2-b76b-452f-9863-5a7355a5c21b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:29.914734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f04c40e2-b76b-452f-9863-5a7355a5c21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:29.976233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1008/2005] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:29.986196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:29.987733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:30.007396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:30.046165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89806152-aee3-4eb4-b304-affba506b117 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:30.050245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe95dca-f3c2-42ce-8b2b-7c7219b9e6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:30.050921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe95dca-f3c2-42ce-8b2b-7c7219b9e6ce service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:30.051197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe95dca-f3c2-42ce-8b2b-7c7219b9e6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:30.051477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe95dca-f3c2-42ce-8b2b-7c7219b9e6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:30.062322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe95dca-f3c2-42ce-8b2b-7c7219b9e6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:30.075922 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe95dca-f3c2-42ce-8b2b-7c7219b9e6ce service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:30.080405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 998/2006] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:30.526370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c40e2-b76b-452f-9863-5a7355a5c21b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:30.527822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f04c40e2-b76b-452f-9863-5a7355a5c21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:30.543146 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c40e2-b76b-452f-9863-5a7355a5c21b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:30.549408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:30.550126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:30.550454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:30.550675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] PATH_INFO: `/v3/projects/8ad2c381df664f8599e059a7af1ae673` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:30.554240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ad2c381df664f8599e059a7af1ae673)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:30.556195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:30.696986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ad2c381df664f8599e059a7af1ae673', 'request_id': 'req-2e91d60a-197d-410f-ae64-378c9137e1c9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:30.702482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ad2c381df664f8599e059a7af1ae673', 'request_id': 'req-2e91d60a-197d-410f-ae64-378c9137e1c9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:30.708630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ad2c381df664f8599e059a7af1ae673', 'request_id': 'req-2e91d60a-197d-410f-ae64-378c9137e1c9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:30.869654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] The token cache is being invalidate because project 8ad2c381df664f8599e059a7af1ae673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:30.869983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:30.870630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2e91d60a-197d-410f-ae64-378c9137e1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:30.872302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1009/2007] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:30 2022] DELETE /identity/v3/projects/8ad2c381df664f8599e059a7af1ae673 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:30.879970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df588a-a35c-4e98-87bb-659b46b60036 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:30.879970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df588a-a35c-4e98-87bb-659b46b60036 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:30.879970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df588a-a35c-4e98-87bb-659b46b60036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:30.899684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f8df588a-a35c-4e98-87bb-659b46b60036 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:30.900920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f8df588a-a35c-4e98-87bb-659b46b60036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:30.972087 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 999/2008] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:30.980092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54880b3-e530-4cc5-8f8b-65336d816d90 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:30.980557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54880b3-e530-4cc5-8f8b-65336d816d90 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:30.980665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54880b3-e530-4cc5-8f8b-65336d816d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:30.988457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:31.007789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e54880b3-e530-4cc5-8f8b-65336d816d90 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:31.007789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e54880b3-e530-4cc5-8f8b-65336d816d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:31.014837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1010/2009] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:31.019220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df588a-a35c-4e98-87bb-659b46b60036 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:31.036790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df588a-a35c-4e98-87bb-659b46b60036 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:31.041630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc3168-4963-4ed9-9f28-68e946dcb1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:31.042520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc3168-4963-4ed9-9f28-68e946dcb1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:31.043021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc3168-4963-4ed9-9f28-68e946dcb1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:31.044590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc3168-4963-4ed9-9f28-68e946dcb1c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:31.045486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc3168-4963-4ed9-9f28-68e946dcb1c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:31.045887 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc3168-4963-4ed9-9f28-68e946dcb1c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:31.047732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc3168-4963-4ed9-9f28-68e946dcb1c5 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:31.064816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1000/2010] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:10:31.066343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54880b3-e530-4cc5-8f8b-65336d816d90 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:31.076563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54880b3-e530-4cc5-8f8b-65336d816d90 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:31.081297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617abe3-3a31-41c2-b1a7-b52c22483ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:31.081956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617abe3-3a31-41c2-b1a7-b52c22483ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:31.082243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617abe3-3a31-41c2-b1a7-b52c22483ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:31.082577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617abe3-3a31-41c2-b1a7-b52c22483ed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:31.083303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617abe3-3a31-41c2-b1a7-b52c22483ed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:31.085872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617abe3-3a31-41c2-b1a7-b52c22483ed2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:31.241095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1011/2011] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:31 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:31.245878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc3168-4963-4ed9-9f28-68e946dcb1c5 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:31.255967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc3168-4963-4ed9-9f28-68e946dcb1c5 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:31.319849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82024f8f-b179-405a-a514-e49629f57279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:31.321318 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82024f8f-b179-405a-a514-e49629f57279 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:31.321613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82024f8f-b179-405a-a514-e49629f57279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:31.321895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82024f8f-b179-405a-a514-e49629f57279 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:31.322625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82024f8f-b179-405a-a514-e49629f57279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:31.324522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82024f8f-b179-405a-a514-e49629f57279 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:31.423816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1001/2012] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:31 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:31.425097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617abe3-3a31-41c2-b1a7-b52c22483ed2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:31.437507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617abe3-3a31-41c2-b1a7-b52c22483ed2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:31.443101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5181c-85f6-4d2f-9b2c-a21878c9f182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:31.443784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5181c-85f6-4d2f-9b2c-a21878c9f182 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:31.444069 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5181c-85f6-4d2f-9b2c-a21878c9f182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:31.444349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5181c-85f6-4d2f-9b2c-a21878c9f182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:31.445153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5181c-85f6-4d2f-9b2c-a21878c9f182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:31.446806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5181c-85f6-4d2f-9b2c-a21878c9f182 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:31.458271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1012/2013] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:10:31.462362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82024f8f-b179-405a-a514-e49629f57279 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:31.472600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82024f8f-b179-405a-a514-e49629f57279 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:31.478214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd584e5-6271-4e18-9ef7-075319137c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:31.478881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd584e5-6271-4e18-9ef7-075319137c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:31.479173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd584e5-6271-4e18-9ef7-075319137c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:31.479453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd584e5-6271-4e18-9ef7-075319137c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:31.480186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd584e5-6271-4e18-9ef7-075319137c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:31.481809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd584e5-6271-4e18-9ef7-075319137c28 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:31.493901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1002/2014] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:31.497915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5181c-85f6-4d2f-9b2c-a21878c9f182 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:31.507740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5181c-85f6-4d2f-9b2c-a21878c9f182 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:31.512994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159a9da-4d38-4241-b7ce-d018bbd8f62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:31.513863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159a9da-4d38-4241-b7ce-d018bbd8f62e admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:31.514145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159a9da-4d38-4241-b7ce-d018bbd8f62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:31.514414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159a9da-4d38-4241-b7ce-d018bbd8f62e admin admin] PATH_INFO: `/v3/projects/2de449548250411c880ed16e0bc75b10/users/3a8296a3d66b4ea2bb318b79bcac87d9/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:31.533348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159a9da-4d38-4241-b7ce-d018bbd8f62e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de449548250411c880ed16e0bc75b10, user_id=3a8296a3d66b4ea2bb318b79bcac87d9, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:31.534171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159a9da-4d38-4241-b7ce-d018bbd8f62e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:31.626550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1013/2015] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:31 2022] PUT /identity/v3/projects/2de449548250411c880ed16e0bc75b10/users/3a8296a3d66b4ea2bb318b79bcac87d9/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:31.630955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae29ff-557d-436e-92ae-58c234529011 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:31.631533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae29ff-557d-436e-92ae-58c234529011 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:31.631919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae29ff-557d-436e-92ae-58c234529011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:31.659923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-97ae29ff-557d-436e-92ae-58c234529011 None None] MFA Rules not processed for user `3a8296a3d66b4ea2bb318b79bcac87d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:31.661344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-97ae29ff-557d-436e-92ae-58c234529011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:31.676913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:31.730089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1003/2016] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:31.741968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159a9da-4d38-4241-b7ce-d018bbd8f62e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:31.744265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e159a9da-4d38-4241-b7ce-d018bbd8f62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:31.807051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159a9da-4d38-4241-b7ce-d018bbd8f62e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:31.810687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a94e61-b0ac-4ff6-bd00-de2d3555d674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:31.812273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a94e61-b0ac-4ff6-bd00-de2d3555d674 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:31.812273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a94e61-b0ac-4ff6-bd00-de2d3555d674 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:31.812273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a94e61-b0ac-4ff6-bd00-de2d3555d674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:31.820617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a94e61-b0ac-4ff6-bd00-de2d3555d674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:31.822449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a94e61-b0ac-4ff6-bd00-de2d3555d674 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:31.832304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1014/2017] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:32.674379 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae29ff-557d-436e-92ae-58c234529011 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:32.675809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-97ae29ff-557d-436e-92ae-58c234529011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:32.743876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae29ff-557d-436e-92ae-58c234529011 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:32.749937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:32.750840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:32.751213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:32.751387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] PATH_INFO: `/v3/users/af45df9a59434a338d36623d6282cf85` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:32.752609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] RBAC: Authorizing `identity:delete_user(user_id=af45df9a59434a338d36623d6282cf85)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:32.754693 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:32.960375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af45df9a59434a338d36623d6282cf85'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:32.969122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af45df9a59434a338d36623d6282cf85'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:32.969819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af45df9a59434a338d36623d6282cf85'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:32.982774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af45df9a59434a338d36623d6282cf85'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:32.983484 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af45df9a59434a338d36623d6282cf85'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:33.049849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1004/2018] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:32 2022] DELETE /identity/v3/users/af45df9a59434a338d36623d6282cf85 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:33.054702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526775c0-ccd3-4d3a-8044-cd6176f874f3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:33.055165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526775c0-ccd3-4d3a-8044-cd6176f874f3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:33.055243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526775c0-ccd3-4d3a-8044-cd6176f874f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:33.075491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-526775c0-ccd3-4d3a-8044-cd6176f874f3 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:33.076759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-526775c0-ccd3-4d3a-8044-cd6176f874f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:33.139372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1015/2019] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:33.150045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:33.151466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:33.188713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:33.211291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c3302-537e-47f5-8859-26072c4c026d admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:33.214968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980c293-b2af-4a50-a5f7-e66148438146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:33.215617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980c293-b2af-4a50-a5f7-e66148438146 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:33.215901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980c293-b2af-4a50-a5f7-e66148438146 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:33.216179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980c293-b2af-4a50-a5f7-e66148438146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:33.225313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980c293-b2af-4a50-a5f7-e66148438146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:33.227226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980c293-b2af-4a50-a5f7-e66148438146 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:33.236824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1005/2020] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:33.631347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526775c0-ccd3-4d3a-8044-cd6176f874f3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:33.632977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-526775c0-ccd3-4d3a-8044-cd6176f874f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:33.647225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526775c0-ccd3-4d3a-8044-cd6176f874f3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:33.653373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:33.658495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:33.658495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:33.658495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] PATH_INFO: `/v3/projects/4b693711a116429abddde9db8816d7bc` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:33.658495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b693711a116429abddde9db8816d7bc)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:33.658495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:33.781855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:33.789098 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b693711a116429abddde9db8816d7bc', 'request_id': 'req-3db387c1-1715-49ce-8669-b8e40138ed6b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:33.794547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b693711a116429abddde9db8816d7bc', 'request_id': 'req-3db387c1-1715-49ce-8669-b8e40138ed6b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:33.800711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b693711a116429abddde9db8816d7bc', 'request_id': 'req-3db387c1-1715-49ce-8669-b8e40138ed6b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:33.884061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] The token cache is being invalidate because project 4b693711a116429abddde9db8816d7bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:33.884381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:33.885042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:33.886821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1016/2021] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:33 2022] DELETE /identity/v3/projects/4b693711a116429abddde9db8816d7bc => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:33.890651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980c293-b2af-4a50-a5f7-e66148438146 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:33.900841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980c293-b2af-4a50-a5f7-e66148438146 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:33.964412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:33.965378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:33.965667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:33.965947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] PATH_INFO: `/v3/users/56505140076943c191c5ce2480533a1d` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:33.967144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=56505140076943c191c5ce2480533a1d)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:33.969126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:34.134428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56505140076943c191c5ce2480533a1d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:34.141324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56505140076943c191c5ce2480533a1d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:34.141974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56505140076943c191c5ce2480533a1d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:34.153929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56505140076943c191c5ce2480533a1d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:34.154543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fdd3292a-23e6-44b3-81cf-efd5fc1de4fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56505140076943c191c5ce2480533a1d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:34.226178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1006/2022] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:33 2022] DELETE /identity/v3/users/56505140076943c191c5ce2480533a1d => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:34.637639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:34.639122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:34.652846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db387c1-1715-49ce-8669-b8e40138ed6b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:34.709406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:34.710310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:34.710599 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:34.711489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] PATH_INFO: `/v3/projects/a22130e5eb144633992405b5cd25c917` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:34.712808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22130e5eb144633992405b5cd25c917)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:34.714875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:34.819025 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22130e5eb144633992405b5cd25c917', 'request_id': 'req-7d5aab39-0d2f-4348-bf65-536b868dd82f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:34.825013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22130e5eb144633992405b5cd25c917', 'request_id': 'req-7d5aab39-0d2f-4348-bf65-536b868dd82f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:34.831379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22130e5eb144633992405b5cd25c917', 'request_id': 'req-7d5aab39-0d2f-4348-bf65-536b868dd82f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:34.906873 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] The token cache is being invalidate because project a22130e5eb144633992405b5cd25c917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:34.907160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:34.907767 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7d5aab39-0d2f-4348-bf65-536b868dd82f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:34.909333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1017/2023] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:34 2022] DELETE /identity/v3/projects/a22130e5eb144633992405b5cd25c917 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:34.914239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02273-ef5e-4d07-9385-0087f4c5161e None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:34.914720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02273-ef5e-4d07-9385-0087f4c5161e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:34.914854 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02273-ef5e-4d07-9385-0087f4c5161e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:34.933938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-89c02273-ef5e-4d07-9385-0087f4c5161e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:34.935122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-89c02273-ef5e-4d07-9385-0087f4c5161e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:34.990193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:35.002237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1007/2024] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:35.010802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94176-2409-4f38-9df8-5b9d182d0c2d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:35.011090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94176-2409-4f38-9df8-5b9d182d0c2d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:35.011368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94176-2409-4f38-9df8-5b9d182d0c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:35.031053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-48b94176-2409-4f38-9df8-5b9d182d0c2d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:35.032261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-48b94176-2409-4f38-9df8-5b9d182d0c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:35.040446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1018/2025] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:35.044937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c02273-ef5e-4d07-9385-0087f4c5161e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:35.055160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c02273-ef5e-4d07-9385-0087f4c5161e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:35.060088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a0e36-b896-4f70-8198-c8c8b5f43e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:35.061031 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a0e36-b896-4f70-8198-c8c8b5f43e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:35.061501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a0e36-b896-4f70-8198-c8c8b5f43e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:35.061884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a0e36-b896-4f70-8198-c8c8b5f43e90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:35.063005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a0e36-b896-4f70-8198-c8c8b5f43e90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:35.063418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a0e36-b896-4f70-8198-c8c8b5f43e90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:35.065309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a0e36-b896-4f70-8198-c8c8b5f43e90 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:35.079344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1008/2026] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:10:35.083582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94176-2409-4f38-9df8-5b9d182d0c2d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:35.098743 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94176-2409-4f38-9df8-5b9d182d0c2d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:35.103454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5378b-a8b8-4a50-b4c6-7184087fbc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:35.104161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5378b-a8b8-4a50-b4c6-7184087fbc49 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:35.104430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5378b-a8b8-4a50-b4c6-7184087fbc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:35.104702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5378b-a8b8-4a50-b4c6-7184087fbc49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:35.105435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5378b-a8b8-4a50-b4c6-7184087fbc49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:35.107346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5378b-a8b8-4a50-b4c6-7184087fbc49 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:35.218466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1019/2027] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:35 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:35.222563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a0e36-b896-4f70-8198-c8c8b5f43e90 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:35.233059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a0e36-b896-4f70-8198-c8c8b5f43e90 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:35.293492 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05800e-38eb-4121-8e79-a8683bb2aca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:35.294543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05800e-38eb-4121-8e79-a8683bb2aca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:35.294655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05800e-38eb-4121-8e79-a8683bb2aca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:35.294866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05800e-38eb-4121-8e79-a8683bb2aca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:35.295598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05800e-38eb-4121-8e79-a8683bb2aca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:35.297656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05800e-38eb-4121-8e79-a8683bb2aca7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:35.365382 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1009/2028] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:35 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:35.370132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5378b-a8b8-4a50-b4c6-7184087fbc49 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:35.383631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5378b-a8b8-4a50-b4c6-7184087fbc49 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:35.388330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975b4f6-4aa7-4345-86fb-f8de58c0b4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:35.389030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975b4f6-4aa7-4345-86fb-f8de58c0b4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:35.389315 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975b4f6-4aa7-4345-86fb-f8de58c0b4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:35.389588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975b4f6-4aa7-4345-86fb-f8de58c0b4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:35.390343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975b4f6-4aa7-4345-86fb-f8de58c0b4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:35.391940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975b4f6-4aa7-4345-86fb-f8de58c0b4d9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:35.402351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1020/2029] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:35.406402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05800e-38eb-4121-8e79-a8683bb2aca7 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:35.417136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05800e-38eb-4121-8e79-a8683bb2aca7 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:35.421693 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dabb2e-e1c8-4e3b-8d67-6cfb351dbecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:35.422336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dabb2e-e1c8-4e3b-8d67-6cfb351dbecc admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:35.422617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dabb2e-e1c8-4e3b-8d67-6cfb351dbecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:35.422888 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dabb2e-e1c8-4e3b-8d67-6cfb351dbecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:35.423601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dabb2e-e1c8-4e3b-8d67-6cfb351dbecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:35.425255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dabb2e-e1c8-4e3b-8d67-6cfb351dbecc admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:35.436007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1010/2030] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:35.440190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975b4f6-4aa7-4345-86fb-f8de58c0b4d9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:35.451326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975b4f6-4aa7-4345-86fb-f8de58c0b4d9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:35.455936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0083a12-6d0b-4ce5-abfc-0beeb9b214c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:35.456849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0083a12-6d0b-4ce5-abfc-0beeb9b214c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:35.457135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0083a12-6d0b-4ce5-abfc-0beeb9b214c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:35.457413 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0083a12-6d0b-4ce5-abfc-0beeb9b214c3 admin admin] PATH_INFO: `/v3/projects/6aee7a1f978641c0be517078b1effd7f/users/deaf77bf08414bdfb04be3a674bb2938/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:35.473798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0083a12-6d0b-4ce5-abfc-0beeb9b214c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aee7a1f978641c0be517078b1effd7f, user_id=deaf77bf08414bdfb04be3a674bb2938, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:35.476499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0083a12-6d0b-4ce5-abfc-0beeb9b214c3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:35.478704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:35.552777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1021/2031] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:35 2022] PUT /identity/v3/projects/6aee7a1f978641c0be517078b1effd7f/users/deaf77bf08414bdfb04be3a674bb2938/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:35.565122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cc764-1678-44c9-b39d-c21af4a9401b None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:35.565504 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cc764-1678-44c9-b39d-c21af4a9401b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:35.565794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cc764-1678-44c9-b39d-c21af4a9401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:35.594726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-511cc764-1678-44c9-b39d-c21af4a9401b None None] MFA Rules not processed for user `deaf77bf08414bdfb04be3a674bb2938`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:35.596014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-511cc764-1678-44c9-b39d-c21af4a9401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:35.667345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1011/2032] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:35.671756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0083a12-6d0b-4ce5-abfc-0beeb9b214c3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:35.734450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0083a12-6d0b-4ce5-abfc-0beeb9b214c3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:35.739191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcb400-72fd-41a6-94cb-c702e339feb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:35.739849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcb400-72fd-41a6-94cb-c702e339feb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:35.740135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcb400-72fd-41a6-94cb-c702e339feb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:35.740412 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcb400-72fd-41a6-94cb-c702e339feb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:35.741215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcb400-72fd-41a6-94cb-c702e339feb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:35.743661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcb400-72fd-41a6-94cb-c702e339feb2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:35.823790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1022/2033] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:35 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:35.827927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cc764-1678-44c9-b39d-c21af4a9401b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:35.839191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cc764-1678-44c9-b39d-c21af4a9401b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:35.888126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:35.897820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd1bdc-34f4-4c3b-b4b2-5db99dd3f1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:35.898484 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd1bdc-34f4-4c3b-b4b2-5db99dd3f1df admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:35.898766 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd1bdc-34f4-4c3b-b4b2-5db99dd3f1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:35.899041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd1bdc-34f4-4c3b-b4b2-5db99dd3f1df admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:35.899724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd1bdc-34f4-4c3b-b4b2-5db99dd3f1df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:35.901662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd1bdc-34f4-4c3b-b4b2-5db99dd3f1df admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:35.994370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1012/2034] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:35 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:35.998802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcb400-72fd-41a6-94cb-c702e339feb2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:36.010018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcb400-72fd-41a6-94cb-c702e339feb2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:36.019367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286213f1-9ebe-404a-81ab-d90c735c6ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:36.020151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286213f1-9ebe-404a-81ab-d90c735c6ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:36.020326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286213f1-9ebe-404a-81ab-d90c735c6ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:36.020611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286213f1-9ebe-404a-81ab-d90c735c6ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:36.028760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286213f1-9ebe-404a-81ab-d90c735c6ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:36.028760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286213f1-9ebe-404a-81ab-d90c735c6ef2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:36.043396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1023/2035] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:10:36.047842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd1bdc-34f4-4c3b-b4b2-5db99dd3f1df admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:36.060387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd1bdc-34f4-4c3b-b4b2-5db99dd3f1df admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:36.065487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd02d6-9eaa-46b0-acb1-afe36f80cd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:36.070074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd02d6-9eaa-46b0-acb1-afe36f80cd4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:36.070074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd02d6-9eaa-46b0-acb1-afe36f80cd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:36.070074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd02d6-9eaa-46b0-acb1-afe36f80cd4e admin admin] PATH_INFO: `/v3/projects/aa7bda0fdd5644459dceaca51909d7dc/users/02823a5b083e43e69f8eaaf3d526e7dc/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:36.088416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd02d6-9eaa-46b0-acb1-afe36f80cd4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7bda0fdd5644459dceaca51909d7dc, user_id=02823a5b083e43e69f8eaaf3d526e7dc, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:36.091128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd02d6-9eaa-46b0-acb1-afe36f80cd4e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:36.174372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1013/2036] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:36 2022] PUT /identity/v3/projects/aa7bda0fdd5644459dceaca51909d7dc/users/02823a5b083e43e69f8eaaf3d526e7dc/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:36.178314 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286213f1-9ebe-404a-81ab-d90c735c6ef2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:36.189021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286213f1-9ebe-404a-81ab-d90c735c6ef2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:36.274618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3303c-8414-416f-b9fb-7339fba02b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:36.275335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3303c-8414-416f-b9fb-7339fba02b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:36.275624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3303c-8414-416f-b9fb-7339fba02b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:36.275953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3303c-8414-416f-b9fb-7339fba02b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:36.277001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3303c-8414-416f-b9fb-7339fba02b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:36.281392 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3303c-8414-416f-b9fb-7339fba02b7f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:36.296325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1024/2037] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:36.297625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd02d6-9eaa-46b0-acb1-afe36f80cd4e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:36.307788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd02d6-9eaa-46b0-acb1-afe36f80cd4e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:36.312494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab12b06-5a80-4109-a08e-d09eecd121e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:36.313408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab12b06-5a80-4109-a08e-d09eecd121e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:36.313688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab12b06-5a80-4109-a08e-d09eecd121e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:36.313963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab12b06-5a80-4109-a08e-d09eecd121e8 admin admin] PATH_INFO: `/v3/projects/aa7bda0fdd5644459dceaca51909d7dc/users/02823a5b083e43e69f8eaaf3d526e7dc/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:36.316298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab12b06-5a80-4109-a08e-d09eecd121e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7bda0fdd5644459dceaca51909d7dc, user_id=02823a5b083e43e69f8eaaf3d526e7dc, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:36.318663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab12b06-5a80-4109-a08e-d09eecd121e8 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:36.329368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1014/2038] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:36 2022] PUT /identity/v3/projects/aa7bda0fdd5644459dceaca51909d7dc/users/02823a5b083e43e69f8eaaf3d526e7dc/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:36.341321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee834e55-5015-4e13-b1af-96461bae8de4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:36.341614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee834e55-5015-4e13-b1af-96461bae8de4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:36.350683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee834e55-5015-4e13-b1af-96461bae8de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:36.370386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ee834e55-5015-4e13-b1af-96461bae8de4 None None] MFA Rules not processed for user `02823a5b083e43e69f8eaaf3d526e7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:36.373763 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ee834e55-5015-4e13-b1af-96461bae8de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:36.429518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:36.446787 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1025/2039] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:36.459023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab12b06-5a80-4109-a08e-d09eecd121e8 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:36.460699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-cab12b06-5a80-4109-a08e-d09eecd121e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:36.529051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab12b06-5a80-4109-a08e-d09eecd121e8 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:36.532869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f01ef-ecd7-408f-b458-eb51af4176d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:36.533468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f01ef-ecd7-408f-b458-eb51af4176d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:36.533756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f01ef-ecd7-408f-b458-eb51af4176d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:36.534040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f01ef-ecd7-408f-b458-eb51af4176d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:36.543972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f01ef-ecd7-408f-b458-eb51af4176d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:36.545493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f01ef-ecd7-408f-b458-eb51af4176d6 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:36.555125 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1015/2040] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:37.492519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee834e55-5015-4e13-b1af-96461bae8de4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:37.502145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee834e55-5015-4e13-b1af-96461bae8de4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:37.506699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8483b-b8fa-4d73-b0b6-25d76195f0a2 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02823a5b083e43e69f8eaaf3d526e7dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa7bda0fdd5644459dceaca51909d7dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:37.507356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8483b-b8fa-4d73-b0b6-25d76195f0a2 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:37.507645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8483b-b8fa-4d73-b0b6-25d76195f0a2 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:37.507925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8483b-b8fa-4d73-b0b6-25d76195f0a2 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:37.509269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8483b-b8fa-4d73-b0b6-25d76195f0a2 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:37.509579 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8483b-b8fa-4d73-b0b6-25d76195f0a2 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:37.511187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8483b-b8fa-4d73-b0b6-25d76195f0a2 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:37.522716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1026/2041] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:10:37.527642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f01ef-ecd7-408f-b458-eb51af4176d6 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:37.536735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f01ef-ecd7-408f-b458-eb51af4176d6 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:37.541824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc5d55-086d-4b40-8e2c-8b0b6df405ca tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02823a5b083e43e69f8eaaf3d526e7dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa7bda0fdd5644459dceaca51909d7dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:37.542480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc5d55-086d-4b40-8e2c-8b0b6df405ca tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:37.542770 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc5d55-086d-4b40-8e2c-8b0b6df405ca tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:37.543050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc5d55-086d-4b40-8e2c-8b0b6df405ca tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:37.543778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc5d55-086d-4b40-8e2c-8b0b6df405ca tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:37.545689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc5d55-086d-4b40-8e2c-8b0b6df405ca tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:37.668489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1016/2042] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:37 2022] POST /identity/v3/projects => generated 354 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:37.672689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8483b-b8fa-4d73-b0b6-25d76195f0a2 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:37.683263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8483b-b8fa-4d73-b0b6-25d76195f0a2 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:37.738173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bc993-1b86-468c-8509-9144cff1cc73 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02823a5b083e43e69f8eaaf3d526e7dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa7bda0fdd5644459dceaca51909d7dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:37.738835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bc993-1b86-468c-8509-9144cff1cc73 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:37.739124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bc993-1b86-468c-8509-9144cff1cc73 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:37.739407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bc993-1b86-468c-8509-9144cff1cc73 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:37.740132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bc993-1b86-468c-8509-9144cff1cc73 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:37.740443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bc993-1b86-468c-8509-9144cff1cc73 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:37.742073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bc993-1b86-468c-8509-9144cff1cc73 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:37.753798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1027/2043] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:10:37.792184 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc5d55-086d-4b40-8e2c-8b0b6df405ca tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:37.802623 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc5d55-086d-4b40-8e2c-8b0b6df405ca tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:37.807713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02823a5b083e43e69f8eaaf3d526e7dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa7bda0fdd5644459dceaca51909d7dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:37.808570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:37.808913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:37.809187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] PATH_INFO: `/v3/projects/ecd5b8c30f5e4667840663e9d5a33b80` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:37.810388 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ecd5b8c30f5e4667840663e9d5a33b80)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:37.813086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:37.947036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd5b8c30f5e4667840663e9d5a33b80', 'request_id': 'req-fb978846-deb4-4f45-b780-1d81c33b7185'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:37.953565 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd5b8c30f5e4667840663e9d5a33b80', 'request_id': 'req-fb978846-deb4-4f45-b780-1d81c33b7185'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:38.004192 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd5b8c30f5e4667840663e9d5a33b80', 'request_id': 'req-fb978846-deb4-4f45-b780-1d81c33b7185'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:38.017864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] The token cache is being invalidate because project ecd5b8c30f5e4667840663e9d5a33b80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:38.018256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:38.018889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fb978846-deb4-4f45-b780-1d81c33b7185 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:38.020423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1017/2044] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:37 2022] DELETE /identity/v3/projects/ecd5b8c30f5e4667840663e9d5a33b80 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:38.590419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bc993-1b86-468c-8509-9144cff1cc73 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:38.592353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8a0bc993-1b86-468c-8509-9144cff1cc73 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:38.667149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bc993-1b86-468c-8509-9144cff1cc73 tempest-VolumeQuotaClassesTest-1597998016 tempest-VolumeQuotaClassesTest-1597998016-project-admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:38.724684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:38.725585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:38.725868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:38.726142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] PATH_INFO: `/v3/users/deaf77bf08414bdfb04be3a674bb2938` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:38.727922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deaf77bf08414bdfb04be3a674bb2938)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:38.729892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:38.750202 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:38.909666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deaf77bf08414bdfb04be3a674bb2938'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:38.916923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deaf77bf08414bdfb04be3a674bb2938'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:38.917583 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deaf77bf08414bdfb04be3a674bb2938'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:38.918183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deaf77bf08414bdfb04be3a674bb2938'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:38.929929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deaf77bf08414bdfb04be3a674bb2938'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:39.010936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1028/2045] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:38 2022] DELETE /identity/v3/users/deaf77bf08414bdfb04be3a674bb2938 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:39.015433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd532f-30fb-41b0-ba34-ccb65b86603f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:39.015984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd532f-30fb-41b0-ba34-ccb65b86603f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:39.016078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd532f-30fb-41b0-ba34-ccb65b86603f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:39.029022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:39.035378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b6dd532f-30fb-41b0-ba34-ccb65b86603f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:39.037052 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b6dd532f-30fb-41b0-ba34-ccb65b86603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:39.096158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1018/2046] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:39.105820 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:39.107491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:39.177501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47f2b8-4be0-4529-b692-bc2ed7e20147 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:39.182656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31beba-5de2-4e79-a1a4-7b5c8cf0b014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:39.183613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31beba-5de2-4e79-a1a4-7b5c8cf0b014 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:39.184048 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31beba-5de2-4e79-a1a4-7b5c8cf0b014 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:39.184469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31beba-5de2-4e79-a1a4-7b5c8cf0b014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:39.197406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31beba-5de2-4e79-a1a4-7b5c8cf0b014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:39.198889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31beba-5de2-4e79-a1a4-7b5c8cf0b014 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:39.208358 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1029/2047] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:39.651433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd532f-30fb-41b0-ba34-ccb65b86603f None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:39.652921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b6dd532f-30fb-41b0-ba34-ccb65b86603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:39.666032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd532f-30fb-41b0-ba34-ccb65b86603f None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:39.671395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:39.672497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:39.673099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:39.673504 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] PATH_INFO: `/v3/projects/6aee7a1f978641c0be517078b1effd7f` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:39.675338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aee7a1f978641c0be517078b1effd7f)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:39.677535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:39.807506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aee7a1f978641c0be517078b1effd7f', 'request_id': 'req-242aaba6-bb41-49f8-a884-a63ed2d317f8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:39.814745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aee7a1f978641c0be517078b1effd7f', 'request_id': 'req-242aaba6-bb41-49f8-a884-a63ed2d317f8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:39.930108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aee7a1f978641c0be517078b1effd7f', 'request_id': 'req-242aaba6-bb41-49f8-a884-a63ed2d317f8'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:39.941855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] The token cache is being invalidate because project 6aee7a1f978641c0be517078b1effd7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:39.942144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:39.942775 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:39.944307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1019/2048] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:39 2022] DELETE /identity/v3/projects/6aee7a1f978641c0be517078b1effd7f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:39.948228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31beba-5de2-4e79-a1a4-7b5c8cf0b014 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:39.959031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31beba-5de2-4e79-a1a4-7b5c8cf0b014 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:40.017811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:40.018711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:40.018995 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:40.019298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] PATH_INFO: `/v3/users/02823a5b083e43e69f8eaaf3d526e7dc` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:40.020547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=02823a5b083e43e69f8eaaf3d526e7dc)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:40.022582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:40.214766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:40.220215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02823a5b083e43e69f8eaaf3d526e7dc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:40.228018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02823a5b083e43e69f8eaaf3d526e7dc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:40.228684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02823a5b083e43e69f8eaaf3d526e7dc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:40.229334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02823a5b083e43e69f8eaaf3d526e7dc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:40.241410 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-816ab937-c43c-4d61-910f-cec50022a1eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02823a5b083e43e69f8eaaf3d526e7dc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:40.316027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1030/2049] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:39 2022] DELETE /identity/v3/users/02823a5b083e43e69f8eaaf3d526e7dc => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:40.788594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:40.790165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:40.803149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242aaba6-bb41-49f8-a884-a63ed2d317f8 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:40.869771 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:40.870704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:40.870991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:40.871269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] PATH_INFO: `/v3/projects/aa7bda0fdd5644459dceaca51909d7dc` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:40.872501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa7bda0fdd5644459dceaca51909d7dc)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:40.874463 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:41.014322 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:41.032080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa7bda0fdd5644459dceaca51909d7dc', 'request_id': 'req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:41.039381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa7bda0fdd5644459dceaca51909d7dc', 'request_id': 'req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:41.112610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa7bda0fdd5644459dceaca51909d7dc', 'request_id': 'req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:41.128664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] The token cache is being invalidate because project aa7bda0fdd5644459dceaca51909d7dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:41.129093 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:41.129723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7134a0e9-a227-43fe-a81f-60a0d7f0ce80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:41.131215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1020/2050] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:40 2022] DELETE /identity/v3/projects/aa7bda0fdd5644459dceaca51909d7dc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:41.138813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5144d-bc08-4653-b48b-1435eb4ee9da None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:41.139273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5144d-bc08-4653-b48b-1435eb4ee9da None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:41.139369 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5144d-bc08-4653-b48b-1435eb4ee9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:41.159355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b8a5144d-bc08-4653-b48b-1435eb4ee9da None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:41.160563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b8a5144d-bc08-4653-b48b-1435eb4ee9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:41.225770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1031/2051] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:41.233808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297ae34-dc48-4e9b-8036-f9acad4bd9de None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:41.234425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297ae34-dc48-4e9b-8036-f9acad4bd9de None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:41.234768 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297ae34-dc48-4e9b-8036-f9acad4bd9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:41.253550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-9297ae34-dc48-4e9b-8036-f9acad4bd9de None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:41.255200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9297ae34-dc48-4e9b-8036-f9acad4bd9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:41.262910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1021/2052] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:41.267244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5144d-bc08-4653-b48b-1435eb4ee9da None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:41.277550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5144d-bc08-4653-b48b-1435eb4ee9da None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:41.292838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831511a-96a2-4b65-97c1-b2c5de55750c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:41.292838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831511a-96a2-4b65-97c1-b2c5de55750c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:41.292838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831511a-96a2-4b65-97c1-b2c5de55750c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:41.292838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831511a-96a2-4b65-97c1-b2c5de55750c admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:41.293285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831511a-96a2-4b65-97c1-b2c5de55750c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:41.293285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831511a-96a2-4b65-97c1-b2c5de55750c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:41.293285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831511a-96a2-4b65-97c1-b2c5de55750c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:41.301864 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1032/2053] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:10:41.306983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297ae34-dc48-4e9b-8036-f9acad4bd9de None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:41.319191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297ae34-dc48-4e9b-8036-f9acad4bd9de None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:41.321785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce2b89-6750-4d8d-b739-01f2b194939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:41.322441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce2b89-6750-4d8d-b739-01f2b194939c admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:41.322715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce2b89-6750-4d8d-b739-01f2b194939c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:41.322994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce2b89-6750-4d8d-b739-01f2b194939c admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:41.323694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce2b89-6750-4d8d-b739-01f2b194939c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:41.325592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce2b89-6750-4d8d-b739-01f2b194939c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:41.502612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1022/2054] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:41 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:41.506679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831511a-96a2-4b65-97c1-b2c5de55750c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:41.516500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831511a-96a2-4b65-97c1-b2c5de55750c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:41.575286 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242690f-3e4c-4880-9a09-e139eae3d51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:41.579236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242690f-3e4c-4880-9a09-e139eae3d51e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:41.579236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242690f-3e4c-4880-9a09-e139eae3d51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:41.579236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242690f-3e4c-4880-9a09-e139eae3d51e admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:41.579236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242690f-3e4c-4880-9a09-e139eae3d51e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:41.579236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242690f-3e4c-4880-9a09-e139eae3d51e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:41.659857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1033/2055] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:41 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:41.661180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce2b89-6750-4d8d-b739-01f2b194939c admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:41.672099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce2b89-6750-4d8d-b739-01f2b194939c admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:41.681910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb4b72-a086-424d-bc86-8f4dbf88e143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:41.681910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb4b72-a086-424d-bc86-8f4dbf88e143 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:41.681910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb4b72-a086-424d-bc86-8f4dbf88e143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:41.681910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb4b72-a086-424d-bc86-8f4dbf88e143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:41.682258 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb4b72-a086-424d-bc86-8f4dbf88e143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:41.682258 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb4b72-a086-424d-bc86-8f4dbf88e143 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:41.691081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1023/2056] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:41.695217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242690f-3e4c-4880-9a09-e139eae3d51e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:41.705129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242690f-3e4c-4880-9a09-e139eae3d51e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:41.709583 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add5278-03e6-4bc3-b64a-b04b83b8e7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:41.710511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add5278-03e6-4bc3-b64a-b04b83b8e7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:41.710511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add5278-03e6-4bc3-b64a-b04b83b8e7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:41.710778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add5278-03e6-4bc3-b64a-b04b83b8e7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:41.712103 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add5278-03e6-4bc3-b64a-b04b83b8e7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:41.713866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add5278-03e6-4bc3-b64a-b04b83b8e7b2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:41.723568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1034/2057] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:41.728503 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb4b72-a086-424d-bc86-8f4dbf88e143 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:41.739285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb4b72-a086-424d-bc86-8f4dbf88e143 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:41.743996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd180b78-20c7-4b95-b70e-8bce964273e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:41.744903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180b78-20c7-4b95-b70e-8bce964273e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:41.745337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180b78-20c7-4b95-b70e-8bce964273e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:41.745620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd180b78-20c7-4b95-b70e-8bce964273e2 admin admin] PATH_INFO: `/v3/projects/1805de5fa9a145a1b137d3f7040ad81a/users/90c0cb27e348479c906391f94b862cff/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:41.762813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd180b78-20c7-4b95-b70e-8bce964273e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1805de5fa9a145a1b137d3f7040ad81a, user_id=90c0cb27e348479c906391f94b862cff, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:41.765568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd180b78-20c7-4b95-b70e-8bce964273e2 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:41.856870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1024/2058] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:41 2022] PUT /identity/v3/projects/1805de5fa9a145a1b137d3f7040ad81a/users/90c0cb27e348479c906391f94b862cff/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:41.867677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b956c6-e2b6-4c45-a25a-af7eb1a4cb64 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:41.868036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b956c6-e2b6-4c45-a25a-af7eb1a4cb64 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:41.868334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b956c6-e2b6-4c45-a25a-af7eb1a4cb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:41.897894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-81b956c6-e2b6-4c45-a25a-af7eb1a4cb64 None None] MFA Rules not processed for user `90c0cb27e348479c906391f94b862cff`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:41.899994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-81b956c6-e2b6-4c45-a25a-af7eb1a4cb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:41.912183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:41.966642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1035/2059] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:41.977730 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd180b78-20c7-4b95-b70e-8bce964273e2 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:41.979526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-dd180b78-20c7-4b95-b70e-8bce964273e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:42.036578 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd180b78-20c7-4b95-b70e-8bce964273e2 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:42.040230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e6a9f-46fa-43e2-957e-d59cbde1e742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:42.041009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e6a9f-46fa-43e2-957e-d59cbde1e742 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:42.041298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e6a9f-46fa-43e2-957e-d59cbde1e742 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:42.041574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e6a9f-46fa-43e2-957e-d59cbde1e742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:42.049973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e6a9f-46fa-43e2-957e-d59cbde1e742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:42.051392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e6a9f-46fa-43e2-957e-d59cbde1e742 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:42.060398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1025/2060] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:42.080302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b956c6-e2b6-4c45-a25a-af7eb1a4cb64 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:42.147091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b956c6-e2b6-4c45-a25a-af7eb1a4cb64 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:42.152507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:42.153419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:42.153699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:42.154003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] PATH_INFO: `/v3/users/90c0cb27e348479c906391f94b862cff` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:42.155263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c0cb27e348479c906391f94b862cff)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:42.157432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:42.346146 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c0cb27e348479c906391f94b862cff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:42.352988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c0cb27e348479c906391f94b862cff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:42.353654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c0cb27e348479c906391f94b862cff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:42.354342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c0cb27e348479c906391f94b862cff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:42.366822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c0cb27e348479c906391f94b862cff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:42.453050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1036/2061] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:42 2022] DELETE /identity/v3/users/90c0cb27e348479c906391f94b862cff => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:42.457600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78966962-58be-46fc-a134-35a7b6d3ecb2 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:42.457898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78966962-58be-46fc-a134-35a7b6d3ecb2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:42.458182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78966962-58be-46fc-a134-35a7b6d3ecb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:42.476034 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-78966962-58be-46fc-a134-35a7b6d3ecb2 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:42.477354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-78966962-58be-46fc-a134-35a7b6d3ecb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:42.528390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:42.536396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1026/2062] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:42.546164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:42.547569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:42.583343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:42.607802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73595fd7-4e89-4b06-aed9-a5d856b3cbdd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:42.608200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92055752-7617-4557-88cd-c9e4026e9819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:42.608874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055752-7617-4557-88cd-c9e4026e9819 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:42.609422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055752-7617-4557-88cd-c9e4026e9819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:42.609698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055752-7617-4557-88cd-c9e4026e9819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:42.619112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92055752-7617-4557-88cd-c9e4026e9819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:42.620625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92055752-7617-4557-88cd-c9e4026e9819 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:42.630261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1037/2063] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:43.064353 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78966962-58be-46fc-a134-35a7b6d3ecb2 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:43.065797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-78966962-58be-46fc-a134-35a7b6d3ecb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:43.078775 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78966962-58be-46fc-a134-35a7b6d3ecb2 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:43.084092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:43.085301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:43.085588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:43.086241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] PATH_INFO: `/v3/users/3875acc8fcc84853a25c89bf934eae2c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:43.090552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3875acc8fcc84853a25c89bf934eae2c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:43.090552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:43.120993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92055752-7617-4557-88cd-c9e4026e9819 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:43.121354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-92055752-7617-4557-88cd-c9e4026e9819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:43.134205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92055752-7617-4557-88cd-c9e4026e9819 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:43.139158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:43.140029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:43.140314 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:43.140638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] PATH_INFO: `/v3/projects/1805de5fa9a145a1b137d3f7040ad81a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:43.141894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1805de5fa9a145a1b137d3f7040ad81a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:43.143847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:43.320199 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3875acc8fcc84853a25c89bf934eae2c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:43.321582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1805de5fa9a145a1b137d3f7040ad81a', 'request_id': 'req-f4d22de9-6e78-49d2-8935-a989d508a79b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:43.327457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1805de5fa9a145a1b137d3f7040ad81a', 'request_id': 'req-f4d22de9-6e78-49d2-8935-a989d508a79b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:43.330319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3875acc8fcc84853a25c89bf934eae2c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:43.331257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3875acc8fcc84853a25c89bf934eae2c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:43.333991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1805de5fa9a145a1b137d3f7040ad81a', 'request_id': 'req-f4d22de9-6e78-49d2-8935-a989d508a79b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:43.345966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3875acc8fcc84853a25c89bf934eae2c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:43.346669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e10b7ba0-77a2-42cd-bdd0-34c1e485c808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3875acc8fcc84853a25c89bf934eae2c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:43.420942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] The token cache is being invalidate because project 1805de5fa9a145a1b137d3f7040ad81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:43.421235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:43.421926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f4d22de9-6e78-49d2-8935-a989d508a79b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:43.423473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1038/2064] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:43 2022] DELETE /identity/v3/projects/1805de5fa9a145a1b137d3f7040ad81a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:10:43.427999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e74a2-10f5-46d7-acfb-2b5ad9c4fe54 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:43.428287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e74a2-10f5-46d7-acfb-2b5ad9c4fe54 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:43.428568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e74a2-10f5-46d7-acfb-2b5ad9c4fe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:43.447300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e66e74a2-10f5-46d7-acfb-2b5ad9c4fe54 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:43.448658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e66e74a2-10f5-46d7-acfb-2b5ad9c4fe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:43.507374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1039/2065] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:43.517022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c3754-3862-4042-9d25-ad72cdd2a916 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:43.517310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c3754-3862-4042-9d25-ad72cdd2a916 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:43.517584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c3754-3862-4042-9d25-ad72cdd2a916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:43.531256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1027/2066] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:43 2022] DELETE /identity/v3/users/3875acc8fcc84853a25c89bf934eae2c => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:43.536532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c26c7d-cade-4277-9b71-ebf0384cabc6 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:43.536854 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c26c7d-cade-4277-9b71-ebf0384cabc6 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:43.537135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c26c7d-cade-4277-9b71-ebf0384cabc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:43.538496 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-696c3754-3862-4042-9d25-ad72cdd2a916 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:43.539719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-696c3754-3862-4042-9d25-ad72cdd2a916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:43.554315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-09c26c7d-cade-4277-9b71-ebf0384cabc6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:43.555528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-09c26c7d-cade-4277-9b71-ebf0384cabc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:43.591681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:43.600407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1040/2067] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:43.605100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c3754-3862-4042-9d25-ad72cdd2a916 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:43.614441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c3754-3862-4042-9d25-ad72cdd2a916 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:43.615348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1028/2068] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:43.619281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8c7bd-a5c2-43ca-84bb-bad47b758689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:43.619928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8c7bd-a5c2-43ca-84bb-bad47b758689 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:43.620214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8c7bd-a5c2-43ca-84bb-bad47b758689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:43.620494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8c7bd-a5c2-43ca-84bb-bad47b758689 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:43.621274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8c7bd-a5c2-43ca-84bb-bad47b758689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:43.621585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8c7bd-a5c2-43ca-84bb-bad47b758689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:43.623184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8c7bd-a5c2-43ca-84bb-bad47b758689 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:43.624713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c26c7d-cade-4277-9b71-ebf0384cabc6 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:43.626303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-09c26c7d-cade-4277-9b71-ebf0384cabc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:43.640701 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1041/2069] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:10:43.641783 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8c7bd-a5c2-43ca-84bb-bad47b758689 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:43.651822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8c7bd-a5c2-43ca-84bb-bad47b758689 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:43.659736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e16b-d845-43d7-8bf6-aec11587d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:43.660620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e16b-d845-43d7-8bf6-aec11587d9ec admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:43.661193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e16b-d845-43d7-8bf6-aec11587d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:43.662791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e16b-d845-43d7-8bf6-aec11587d9ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:43.663622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4e16b-d845-43d7-8bf6-aec11587d9ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:43.665693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4e16b-d845-43d7-8bf6-aec11587d9ec admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:43.691655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c26c7d-cade-4277-9b71-ebf0384cabc6 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:43.695490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddf220-ab28-4125-942d-93ac93430c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:43.696674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddf220-ab28-4125-942d-93ac93430c9b service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:43.696999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddf220-ab28-4125-942d-93ac93430c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:43.697328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddf220-ab28-4125-942d-93ac93430c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:43.706567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddf220-ab28-4125-942d-93ac93430c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:43.708007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddf220-ab28-4125-942d-93ac93430c9b service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:43.718126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1029/2070] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:10:43.799272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1042/2071] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:43 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:43.803088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddf220-ab28-4125-942d-93ac93430c9b service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:43.814053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddf220-ab28-4125-942d-93ac93430c9b service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:43.826551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:43.871032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa4e0b-a7f8-425e-9859-adead458e481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:43.871697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa4e0b-a7f8-425e-9859-adead458e481 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:43.871986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa4e0b-a7f8-425e-9859-adead458e481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:43.872268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa4e0b-a7f8-425e-9859-adead458e481 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:43.873006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa4e0b-a7f8-425e-9859-adead458e481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:43.874920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa4e0b-a7f8-425e-9859-adead458e481 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:44.073691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1030/2072] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:43 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:44.077962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e16b-d845-43d7-8bf6-aec11587d9ec admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:44.088976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e16b-d845-43d7-8bf6-aec11587d9ec admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:44.093838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02548c92-96ec-4425-ab78-1b9d8eeee243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:44.094511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02548c92-96ec-4425-ab78-1b9d8eeee243 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:44.094797 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02548c92-96ec-4425-ab78-1b9d8eeee243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:44.102082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02548c92-96ec-4425-ab78-1b9d8eeee243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:44.102082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02548c92-96ec-4425-ab78-1b9d8eeee243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:44.102082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02548c92-96ec-4425-ab78-1b9d8eeee243 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:44.108117 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1043/2073] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:44.112734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa4e0b-a7f8-425e-9859-adead458e481 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:44.123868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa4e0b-a7f8-425e-9859-adead458e481 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:44.128797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6559b6f-9c5b-4b95-9aab-0a4f68dc6de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:44.129837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6559b6f-9c5b-4b95-9aab-0a4f68dc6de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:44.129837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6559b6f-9c5b-4b95-9aab-0a4f68dc6de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:44.131591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6559b6f-9c5b-4b95-9aab-0a4f68dc6de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:44.131591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6559b6f-9c5b-4b95-9aab-0a4f68dc6de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:44.132816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6559b6f-9c5b-4b95-9aab-0a4f68dc6de9 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:44.144101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1031/2074] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:44.148262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02548c92-96ec-4425-ab78-1b9d8eeee243 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:44.151932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6559b6f-9c5b-4b95-9aab-0a4f68dc6de9 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:44.155805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e6559b6f-9c5b-4b95-9aab-0a4f68dc6de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:44.158772 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02548c92-96ec-4425-ab78-1b9d8eeee243 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:44.165934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebdeb3-20dd-4afa-8ee0-558e476f7384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:44.166816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebdeb3-20dd-4afa-8ee0-558e476f7384 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:44.167096 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebdeb3-20dd-4afa-8ee0-558e476f7384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:44.167367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebdeb3-20dd-4afa-8ee0-558e476f7384 admin admin] PATH_INFO: `/v3/projects/398968bb78824dc697ace2d523bc885e/users/337ad1eb42c04f59b5b5e096b43c0c97/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:44.170117 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6559b6f-9c5b-4b95-9aab-0a4f68dc6de9 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:44.174608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:44.175460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:44.175739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:44.176228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] PATH_INFO: `/v3/users/3a8296a3d66b4ea2bb318b79bcac87d9` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:44.177486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8296a3d66b4ea2bb318b79bcac87d9)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:44.181037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:44.183597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebdeb3-20dd-4afa-8ee0-558e476f7384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=398968bb78824dc697ace2d523bc885e, user_id=337ad1eb42c04f59b5b5e096b43c0c97, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:44.186335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebdeb3-20dd-4afa-8ee0-558e476f7384 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:44.316272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1044/2075] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:44 2022] PUT /identity/v3/projects/398968bb78824dc697ace2d523bc885e/users/337ad1eb42c04f59b5b5e096b43c0c97/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:44.317138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebdeb3-20dd-4afa-8ee0-558e476f7384 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:44.318734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a0ebdeb3-20dd-4afa-8ee0-558e476f7384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:44.385240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebdeb3-20dd-4afa-8ee0-558e476f7384 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:44.391154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:44.392036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:44.392317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:44.392590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] PATH_INFO: `/v3/projects/39a3d9d44e7044d1a3d6fb780e0f79db` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:44.393826 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39a3d9d44e7044d1a3d6fb780e0f79db)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:44.394182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8296a3d66b4ea2bb318b79bcac87d9'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.395869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:44.401274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8296a3d66b4ea2bb318b79bcac87d9'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.401940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8296a3d66b4ea2bb318b79bcac87d9'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.415603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8296a3d66b4ea2bb318b79bcac87d9'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.416659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-38eaa144-c977-4a14-85ed-848bbd4c1770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8296a3d66b4ea2bb318b79bcac87d9'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.490789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1032/2076] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:44 2022] DELETE /identity/v3/users/3a8296a3d66b4ea2bb318b79bcac87d9 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:10:44.492341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c463-3c1e-4e1a-9d04-404319be8111 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:44.493391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c463-3c1e-4e1a-9d04-404319be8111 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:44.495652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740c463-3c1e-4e1a-9d04-404319be8111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:44.509765 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a3d9d44e7044d1a3d6fb780e0f79db', 'request_id': 'req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.526287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a3d9d44e7044d1a3d6fb780e0f79db', 'request_id': 'req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.529710 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-7740c463-3c1e-4e1a-9d04-404319be8111 None None] MFA Rules not processed for user `337ad1eb42c04f59b5b5e096b43c0c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:44.530934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7740c463-3c1e-4e1a-9d04-404319be8111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:44.535902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a3d9d44e7044d1a3d6fb780e0f79db', 'request_id': 'req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.599897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:44.609694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1033/2077] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:44.611022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0fc03-65ba-44ae-a7df-6ef0fecdb6f4 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:44.611308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0fc03-65ba-44ae-a7df-6ef0fecdb6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:44.611579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0fc03-65ba-44ae-a7df-6ef0fecdb6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:44.616931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] The token cache is being invalidate because project 39a3d9d44e7044d1a3d6fb780e0f79db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:44.617274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.618143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.619949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1045/2078] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:44 2022] DELETE /identity/v3/projects/39a3d9d44e7044d1a3d6fb780e0f79db => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:44.623776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:44.625348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:44.633032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-0ce0fc03-65ba-44ae-a7df-6ef0fecdb6f4 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:44.634342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0ce0fc03-65ba-44ae-a7df-6ef0fecdb6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:44.649836 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:44.696677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009f57a-c6a2-42a1-9a49-5f4af50f33f5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:44.702113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:44.702560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1034/2079] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:44.703129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:44.703435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0fc03-65ba-44ae-a7df-6ef0fecdb6f4 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:44.703539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:44.703710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] PATH_INFO: `/v3/users/27fc61eaf82647fbbecc138042bf795a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:44.704743 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0ce0fc03-65ba-44ae-a7df-6ef0fecdb6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:44.704996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] RBAC: Authorizing `identity:delete_user(user_id=27fc61eaf82647fbbecc138042bf795a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:44.711117 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:44.761005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0fc03-65ba-44ae-a7df-6ef0fecdb6f4 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:44.764433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e82a7-92f9-4e59-99ff-8bdbf388c0d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:44.765106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e82a7-92f9-4e59-99ff-8bdbf388c0d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:44.765390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e82a7-92f9-4e59-99ff-8bdbf388c0d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:44.765733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e82a7-92f9-4e59-99ff-8bdbf388c0d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:44.767435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-200e82a7-92f9-4e59-99ff-8bdbf388c0d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:44.778594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e82a7-92f9-4e59-99ff-8bdbf388c0d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:44.780007 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e82a7-92f9-4e59-99ff-8bdbf388c0d7 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:44.788657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1035/2080] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:44.790017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e82a7-92f9-4e59-99ff-8bdbf388c0d7 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:44.791335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-200e82a7-92f9-4e59-99ff-8bdbf388c0d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:44.851008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e82a7-92f9-4e59-99ff-8bdbf388c0d7 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:44.855457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c2559-eafc-4f50-a8a0-679ebc2b798f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:44.856110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c2559-eafc-4f50-a8a0-679ebc2b798f service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:44.856401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c2559-eafc-4f50-a8a0-679ebc2b798f service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:44.856687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c2559-eafc-4f50-a8a0-679ebc2b798f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:44.867609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c2559-eafc-4f50-a8a0-679ebc2b798f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:44.869068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c2559-eafc-4f50-a8a0-679ebc2b798f service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:44.878818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1036/2081] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:44.933249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27fc61eaf82647fbbecc138042bf795a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.940237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27fc61eaf82647fbbecc138042bf795a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.941340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27fc61eaf82647fbbecc138042bf795a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.942036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27fc61eaf82647fbbecc138042bf795a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:44.956206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27fc61eaf82647fbbecc138042bf795a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.054223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1046/2082] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:10:44 2022] DELETE /identity/v3/users/27fc61eaf82647fbbecc138042bf795a => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:45.366396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c2559-eafc-4f50-a8a0-679ebc2b798f service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:45.367847 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5d6c2559-eafc-4f50-a8a0-679ebc2b798f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:45.390520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c2559-eafc-4f50-a8a0-679ebc2b798f service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:45.454231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:45.455089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:45.455371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:45.455650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] PATH_INFO: `/v3/projects/2de449548250411c880ed16e0bc75b10` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:45.456840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de449548250411c880ed16e0bc75b10)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:45.458750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:45.564580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:45.566407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:45.597165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8c2e6-e5c2-43d2-8fb4-f5779b3e488e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:45.608918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de449548250411c880ed16e0bc75b10', 'request_id': 'req-fe783557-84b1-465c-a822-38c452eb2847'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.615972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de449548250411c880ed16e0bc75b10', 'request_id': 'req-fe783557-84b1-465c-a822-38c452eb2847'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.685809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:45.686545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:45.686832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:45.687106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] PATH_INFO: `/v3/projects/a4e32fc3af1e48d687ec84179716146a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:45.688480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e32fc3af1e48d687ec84179716146a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:45.690473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:45.736621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:45.737649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de449548250411c880ed16e0bc75b10', 'request_id': 'req-fe783557-84b1-465c-a822-38c452eb2847'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.749221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] The token cache is being invalidate because project 2de449548250411c880ed16e0bc75b10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:45.749513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.750209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-fe783557-84b1-465c-a822-38c452eb2847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.751991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1037/2083] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:45 2022] DELETE /identity/v3/projects/2de449548250411c880ed16e0bc75b10 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:45.756819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a5285-ddc9-4d6f-ba1d-4958a0bc7b21 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:45.757092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a5285-ddc9-4d6f-ba1d-4958a0bc7b21 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:45.757506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a5285-ddc9-4d6f-ba1d-4958a0bc7b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:45.777236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-9d4a5285-ddc9-4d6f-ba1d-4958a0bc7b21 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:45.778882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9d4a5285-ddc9-4d6f-ba1d-4958a0bc7b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:45.788582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1038/2084] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:45.796481 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6af8e-e7c7-44d0-8018-bb8ec3ae3faa None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:45.796706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6af8e-e7c7-44d0-8018-bb8ec3ae3faa None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:45.796942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6af8e-e7c7-44d0-8018-bb8ec3ae3faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:45.814957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-9ee6af8e-e7c7-44d0-8018-bb8ec3ae3faa None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:45.816125 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9ee6af8e-e7c7-44d0-8018-bb8ec3ae3faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:45.821527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e32fc3af1e48d687ec84179716146a', 'request_id': 'req-5e27511e-9828-4c0d-836b-2778865f870c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.827718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e32fc3af1e48d687ec84179716146a', 'request_id': 'req-5e27511e-9828-4c0d-836b-2778865f870c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.834080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e32fc3af1e48d687ec84179716146a', 'request_id': 'req-5e27511e-9828-4c0d-836b-2778865f870c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.877696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1039/2085] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:45.881949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6af8e-e7c7-44d0-8018-bb8ec3ae3faa None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:45.891026 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6af8e-e7c7-44d0-8018-bb8ec3ae3faa None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:45.895649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00504c4-dd41-488a-abda-98d143f3d589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:45.899009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00504c4-dd41-488a-abda-98d143f3d589 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:45.899009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00504c4-dd41-488a-abda-98d143f3d589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:45.899009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00504c4-dd41-488a-abda-98d143f3d589 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:45.899009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00504c4-dd41-488a-abda-98d143f3d589 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:45.899009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00504c4-dd41-488a-abda-98d143f3d589 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:45.899444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00504c4-dd41-488a-abda-98d143f3d589 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:45.911487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1040/2086] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:10:45.915491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00504c4-dd41-488a-abda-98d143f3d589 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:45.924682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00504c4-dd41-488a-abda-98d143f3d589 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:45.929393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da35c19-8869-4bc7-91c8-de5157d5070e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:45.934574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da35c19-8869-4bc7-91c8-de5157d5070e admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:45.934574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da35c19-8869-4bc7-91c8-de5157d5070e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:45.934574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da35c19-8869-4bc7-91c8-de5157d5070e admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:45.934574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da35c19-8869-4bc7-91c8-de5157d5070e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:45.934574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da35c19-8869-4bc7-91c8-de5157d5070e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:45.954575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:45.959348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] The token cache is being invalidate because project a4e32fc3af1e48d687ec84179716146a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:10:45.959649 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.960258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5e27511e-9828-4c0d-836b-2778865f870c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:10:45.961863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1047/2087] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:10:45 2022] DELETE /identity/v3/projects/a4e32fc3af1e48d687ec84179716146a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:45.967314 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962789b2-f1ed-4737-81ae-b5463bbc31cd None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:45.967607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962789b2-f1ed-4737-81ae-b5463bbc31cd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:45.967893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962789b2-f1ed-4737-81ae-b5463bbc31cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:45.986180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-962789b2-f1ed-4737-81ae-b5463bbc31cd None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:45.987362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-962789b2-f1ed-4737-81ae-b5463bbc31cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:45.995991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1048/2088] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:46.004684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfcb3f-8697-4175-9402-077fd30bcfa9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.005011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfcb3f-8697-4175-9402-077fd30bcfa9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.005289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfcb3f-8697-4175-9402-077fd30bcfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.024513 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-abdfcb3f-8697-4175-9402-077fd30bcfa9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:46.025804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-abdfcb3f-8697-4175-9402-077fd30bcfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:46.033669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1049/2089] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:46.037628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfcb3f-8697-4175-9402-077fd30bcfa9 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.044405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1041/2090] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:45 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:46.045543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da35c19-8869-4bc7-91c8-de5157d5070e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.047284 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2da35c19-8869-4bc7-91c8-de5157d5070e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:46.048467 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfcb3f-8697-4175-9402-077fd30bcfa9 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.094091 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:46.104894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215b70d-7a38-46ea-95e3-812b8f65f095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.105581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215b70d-7a38-46ea-95e3-812b8f65f095 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.105852 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215b70d-7a38-46ea-95e3-812b8f65f095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.106136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215b70d-7a38-46ea-95e3-812b8f65f095 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.106873 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215b70d-7a38-46ea-95e3-812b8f65f095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:10:46.107183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215b70d-7a38-46ea-95e3-812b8f65f095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.108915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215b70d-7a38-46ea-95e3-812b8f65f095 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.112267 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da35c19-8869-4bc7-91c8-de5157d5070e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.117121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c1a91-bbf3-431d-9bdb-a0d1dcdc33a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.117842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c1a91-bbf3-431d-9bdb-a0d1dcdc33a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.118159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c1a91-bbf3-431d-9bdb-a0d1dcdc33a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.118489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c1a91-bbf3-431d-9bdb-a0d1dcdc33a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.119891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c1a91-bbf3-431d-9bdb-a0d1dcdc33a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.121433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1050/2091] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:10:46.121849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c1a91-bbf3-431d-9bdb-a0d1dcdc33a8 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.125521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215b70d-7a38-46ea-95e3-812b8f65f095 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.135689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215b70d-7a38-46ea-95e3-812b8f65f095 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.140398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb40ce-faa9-46a0-afc2-ceb0d26372bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.141065 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb40ce-faa9-46a0-afc2-ceb0d26372bd admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.141410 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb40ce-faa9-46a0-afc2-ceb0d26372bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.141699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb40ce-faa9-46a0-afc2-ceb0d26372bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.142387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb40ce-faa9-46a0-afc2-ceb0d26372bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.144281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb40ce-faa9-46a0-afc2-ceb0d26372bd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.238424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1042/2092] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:46 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:46.242371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c1a91-bbf3-431d-9bdb-a0d1dcdc33a8 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.252180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c1a91-bbf3-431d-9bdb-a0d1dcdc33a8 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.256748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff51a6-6ef1-4ce3-b8b2-ec1506b3342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.257414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff51a6-6ef1-4ce3-b8b2-ec1506b3342a admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.257704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff51a6-6ef1-4ce3-b8b2-ec1506b3342a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.257986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff51a6-6ef1-4ce3-b8b2-ec1506b3342a admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.258703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff51a6-6ef1-4ce3-b8b2-ec1506b3342a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.260876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff51a6-6ef1-4ce3-b8b2-ec1506b3342a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.271199 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1043/2094] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:10:46.272728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1051/2093] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:46 2022] POST /identity/v3/projects => generated 409 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:10:46.273326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff51a6-6ef1-4ce3-b8b2-ec1506b3342a admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.275041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb40ce-faa9-46a0-afc2-ceb0d26372bd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.284294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff51a6-6ef1-4ce3-b8b2-ec1506b3342a admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.339006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb40ce-faa9-46a0-afc2-ceb0d26372bd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.344026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04469f8-5088-49ac-8016-a42f8a4c3a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.344628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff10b1b-9fe5-4fc5-9074-7c4bfaddb957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.344801 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04469f8-5088-49ac-8016-a42f8a4c3a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.345248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04469f8-5088-49ac-8016-a42f8a4c3a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.345358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff10b1b-9fe5-4fc5-9074-7c4bfaddb957 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.345532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04469f8-5088-49ac-8016-a42f8a4c3a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.345635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff10b1b-9fe5-4fc5-9074-7c4bfaddb957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.345938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff10b1b-9fe5-4fc5-9074-7c4bfaddb957 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.346298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04469f8-5088-49ac-8016-a42f8a4c3a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.346628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff10b1b-9fe5-4fc5-9074-7c4bfaddb957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.347909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04469f8-5088-49ac-8016-a42f8a4c3a0c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.348537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff10b1b-9fe5-4fc5-9074-7c4bfaddb957 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.362335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1052/2095] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:10:46.363345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04469f8-5088-49ac-8016-a42f8a4c3a0c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.373924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04469f8-5088-49ac-8016-a42f8a4c3a0c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.378544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31914112-98e4-41c8-8598-b09f20612cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.379420 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31914112-98e4-41c8-8598-b09f20612cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.379721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31914112-98e4-41c8-8598-b09f20612cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.380015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31914112-98e4-41c8-8598-b09f20612cdd admin admin] PATH_INFO: `/v3/projects/e98dfd576b4d465095388846eff5da5f/users/8dee1a69a2ec4043b17a59613b1e8f83/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.394465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31914112-98e4-41c8-8598-b09f20612cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98dfd576b4d465095388846eff5da5f, user_id=8dee1a69a2ec4043b17a59613b1e8f83, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.397162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31914112-98e4-41c8-8598-b09f20612cdd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.441946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1044/2096] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:46 2022] POST /identity/v3/users => generated 305 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:46.442932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff10b1b-9fe5-4fc5-9074-7c4bfaddb957 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.452550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff10b1b-9fe5-4fc5-9074-7c4bfaddb957 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.457313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6b2be3-b3df-402c-a3e2-64f988085442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.457954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b2be3-b3df-402c-a3e2-64f988085442 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.458241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b2be3-b3df-402c-a3e2-64f988085442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.458518 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6b2be3-b3df-402c-a3e2-64f988085442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.459237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b2be3-b3df-402c-a3e2-64f988085442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.460846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6b2be3-b3df-402c-a3e2-64f988085442 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.467078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1053/2097] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:46 2022] PUT /identity/v3/projects/e98dfd576b4d465095388846eff5da5f/users/8dee1a69a2ec4043b17a59613b1e8f83/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:46.471000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1045/2098] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:10:46.474876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31914112-98e4-41c8-8598-b09f20612cdd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.481098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425900-c933-4574-8a19-f1b246ad9626 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.481666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425900-c933-4574-8a19-f1b246ad9626 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.481979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425900-c933-4574-8a19-f1b246ad9626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.509196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-06425900-c933-4574-8a19-f1b246ad9626 None None] MFA Rules not processed for user `8dee1a69a2ec4043b17a59613b1e8f83`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:46.510401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-06425900-c933-4574-8a19-f1b246ad9626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:46.536879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31914112-98e4-41c8-8598-b09f20612cdd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.542043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7efcfa-cc5a-4572-9773-548ce4460f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.542754 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7efcfa-cc5a-4572-9773-548ce4460f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.543040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7efcfa-cc5a-4572-9773-548ce4460f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.543323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7efcfa-cc5a-4572-9773-548ce4460f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.544071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7efcfa-cc5a-4572-9773-548ce4460f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.545689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7efcfa-cc5a-4572-9773-548ce4460f7a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.555790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1054/2099] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:46.560356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7efcfa-cc5a-4572-9773-548ce4460f7a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.570147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7efcfa-cc5a-4572-9773-548ce4460f7a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.575001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799ec24-5a5e-44a5-9199-ec37b2863941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.575426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1046/2100] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:46.576002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799ec24-5a5e-44a5-9199-ec37b2863941 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.576292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799ec24-5a5e-44a5-9199-ec37b2863941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.576615 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799ec24-5a5e-44a5-9199-ec37b2863941 admin admin] PATH_INFO: `/v3/projects/6f629484a82d4b8b82827acb08c4be5f/users/590f50a44aa540d181aa8cb5b55cbeef/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.586685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06425900-c933-4574-8a19-f1b246ad9626 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.587318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:46.588261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-06425900-c933-4574-8a19-f1b246ad9626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:46.595968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799ec24-5a5e-44a5-9199-ec37b2863941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f629484a82d4b8b82827acb08c4be5f, user_id=590f50a44aa540d181aa8cb5b55cbeef, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.598865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799ec24-5a5e-44a5-9199-ec37b2863941 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.616884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:46.647329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06425900-c933-4574-8a19-f1b246ad9626 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.650976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd953760-5d95-4d91-abe9-406a4e2f2da8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.651619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd953760-5d95-4d91-abe9-406a4e2f2da8 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.651899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd953760-5d95-4d91-abe9-406a4e2f2da8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.652178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd953760-5d95-4d91-abe9-406a4e2f2da8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.660207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd953760-5d95-4d91-abe9-406a4e2f2da8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.661660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd953760-5d95-4d91-abe9-406a4e2f2da8 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.670881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1047/2101] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:46.675661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1055/2102] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:46 2022] PUT /identity/v3/projects/6f629484a82d4b8b82827acb08c4be5f/users/590f50a44aa540d181aa8cb5b55cbeef/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:10:46.691470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623f4a1-5f68-43a0-85a5-368f948302fd None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.691470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623f4a1-5f68-43a0-85a5-368f948302fd None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.691470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623f4a1-5f68-43a0-85a5-368f948302fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.714454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1623f4a1-5f68-43a0-85a5-368f948302fd None None] MFA Rules not processed for user `590f50a44aa540d181aa8cb5b55cbeef`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:46.715689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1623f4a1-5f68-43a0-85a5-368f948302fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:46.779713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1048/2103] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:46 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:10:46.784247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799ec24-5a5e-44a5-9199-ec37b2863941 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.851495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799ec24-5a5e-44a5-9199-ec37b2863941 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:46.852823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cac64-566d-4367-860c-77078ea244ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:46.854138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cac64-566d-4367-860c-77078ea244ce admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:46.854423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cac64-566d-4367-860c-77078ea244ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:46.854696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cac64-566d-4367-860c-77078ea244ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:46.855415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6cac64-566d-4367-860c-77078ea244ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:46.857389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6cac64-566d-4367-860c-77078ea244ce admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:46.960025 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1056/2104] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:10:46 2022] POST /identity/v3/projects => generated 409 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:46.964756 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623f4a1-5f68-43a0-85a5-368f948302fd None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:46.976456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623f4a1-5f68-43a0-85a5-368f948302fd None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:47.038881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75839bec-0f74-4ddc-9b63-e90fb9cdaa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:47.039562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75839bec-0f74-4ddc-9b63-e90fb9cdaa63 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:47.039850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75839bec-0f74-4ddc-9b63-e90fb9cdaa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:47.040129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75839bec-0f74-4ddc-9b63-e90fb9cdaa63 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:47.041801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75839bec-0f74-4ddc-9b63-e90fb9cdaa63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:47.044798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75839bec-0f74-4ddc-9b63-e90fb9cdaa63 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:47.187142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1049/2105] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:10:46 2022] POST /identity/v3/users => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:10:47.191243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cac64-566d-4367-860c-77078ea244ce admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:47.202230 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cac64-566d-4367-860c-77078ea244ce admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:47.206921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104a0c8-f120-41fd-9fd7-b4d5d9603e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:47.207588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104a0c8-f120-41fd-9fd7-b4d5d9603e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:47.207876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104a0c8-f120-41fd-9fd7-b4d5d9603e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:47.208165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104a0c8-f120-41fd-9fd7-b4d5d9603e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:47.208924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104a0c8-f120-41fd-9fd7-b4d5d9603e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:47.210540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104a0c8-f120-41fd-9fd7-b4d5d9603e5a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:47.224049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1057/2106] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:10:47.228269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75839bec-0f74-4ddc-9b63-e90fb9cdaa63 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:47.238466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75839bec-0f74-4ddc-9b63-e90fb9cdaa63 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:47.243146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c12af6-ca93-4ba3-a083-5f4c0f2599ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:47.244036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c12af6-ca93-4ba3-a083-5f4c0f2599ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:47.244312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c12af6-ca93-4ba3-a083-5f4c0f2599ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:47.244659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c12af6-ca93-4ba3-a083-5f4c0f2599ad admin admin] PATH_INFO: `/v3/projects/f067e36406b1473ebc3057f8b1c9cd7f/users/cd8eeb28a8a348ac8ef5605c111265ef/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:47.259716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c12af6-ca93-4ba3-a083-5f4c0f2599ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f067e36406b1473ebc3057f8b1c9cd7f, user_id=cd8eeb28a8a348ac8ef5605c111265ef, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:47.262468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c12af6-ca93-4ba3-a083-5f4c0f2599ad admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:47.383472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1050/2107] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:47 2022] PUT /identity/v3/projects/f067e36406b1473ebc3057f8b1c9cd7f/users/cd8eeb28a8a348ac8ef5605c111265ef/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:10:47.387849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104a0c8-f120-41fd-9fd7-b4d5d9603e5a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:47.397867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104a0c8-f120-41fd-9fd7-b4d5d9603e5a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:47.443425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:47.457515 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c6f88-0016-42c9-b2da-12a0593a9202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:47.458222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c6f88-0016-42c9-b2da-12a0593a9202 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:47.458514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c6f88-0016-42c9-b2da-12a0593a9202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:47.459655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c6f88-0016-42c9-b2da-12a0593a9202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:47.459655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c6f88-0016-42c9-b2da-12a0593a9202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:47.461302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c6f88-0016-42c9-b2da-12a0593a9202 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:47.471970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1058/2108] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:10:47.476103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c12af6-ca93-4ba3-a083-5f4c0f2599ad admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:47.486811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c12af6-ca93-4ba3-a083-5f4c0f2599ad admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:47.491824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b157acf-81e5-420c-a6b0-230f52685ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:47.492692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b157acf-81e5-420c-a6b0-230f52685ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:47.493004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b157acf-81e5-420c-a6b0-230f52685ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:47.493278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b157acf-81e5-420c-a6b0-230f52685ebc admin admin] PATH_INFO: `/v3/projects/f067e36406b1473ebc3057f8b1c9cd7f/users/cd8eeb28a8a348ac8ef5605c111265ef/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:47.495728 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b157acf-81e5-420c-a6b0-230f52685ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f067e36406b1473ebc3057f8b1c9cd7f, user_id=cd8eeb28a8a348ac8ef5605c111265ef, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:47.498437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b157acf-81e5-420c-a6b0-230f52685ebc admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:47.512423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1051/2109] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:10:47 2022] PUT /identity/v3/projects/f067e36406b1473ebc3057f8b1c9cd7f/users/cd8eeb28a8a348ac8ef5605c111265ef/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:10:47.522978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f4703-9393-4b13-b8aa-2ecdce7ec8ab None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:47.523270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f4703-9393-4b13-b8aa-2ecdce7ec8ab None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:47.523548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f4703-9393-4b13-b8aa-2ecdce7ec8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:47.550219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-232f4703-9393-4b13-b8aa-2ecdce7ec8ab None None] MFA Rules not processed for user `cd8eeb28a8a348ac8ef5605c111265ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:10:47.551441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-232f4703-9393-4b13-b8aa-2ecdce7ec8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:47.620460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1059/2110] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:10:47 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:10:47.630938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b157acf-81e5-420c-a6b0-230f52685ebc admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:47.632642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1b157acf-81e5-420c-a6b0-230f52685ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:10:47.639064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:47.693171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b157acf-81e5-420c-a6b0-230f52685ebc admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:47.697925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9fef7-c834-44a1-861e-cc961d757111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:47.698495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9fef7-c834-44a1-861e-cc961d757111 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:47.698807 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9fef7-c834-44a1-861e-cc961d757111 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:47.699091 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9fef7-c834-44a1-861e-cc961d757111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:47.708143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9fef7-c834-44a1-861e-cc961d757111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:47.709617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9fef7-c834-44a1-861e-cc961d757111 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:47.718885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1052/2111] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:47 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:48.052527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f4703-9393-4b13-b8aa-2ecdce7ec8ab None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:10:48.062586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f4703-9393-4b13-b8aa-2ecdce7ec8ab None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:10:48.066925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9af162-f2bf-4d84-ad13-eb49bbb5153a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:10:48.067610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9af162-f2bf-4d84-ad13-eb49bbb5153a service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:10:48.067895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9af162-f2bf-4d84-ad13-eb49bbb5153a service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:10:48.068165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9af162-f2bf-4d84-ad13-eb49bbb5153a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:10:48.129742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9af162-f2bf-4d84-ad13-eb49bbb5153a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:10:48.131201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9af162-f2bf-4d84-ad13-eb49bbb5153a service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:10:48.140307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1060/2112] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:10:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:10:57.757862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=25 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:10:58.136222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=70 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:04.351391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9fef7-c834-44a1-861e-cc961d757111 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:04.422595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9fef7-c834-44a1-861e-cc961d757111 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:04.427496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:04.428401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:04.428683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:04.428987 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] PATH_INFO: `/v3/users/590f50a44aa540d181aa8cb5b55cbeef` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:04.430205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=590f50a44aa540d181aa8cb5b55cbeef)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:04.432134 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:04.626238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590f50a44aa540d181aa8cb5b55cbeef'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:04.633242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590f50a44aa540d181aa8cb5b55cbeef'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:04.634500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590f50a44aa540d181aa8cb5b55cbeef'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:04.646385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590f50a44aa540d181aa8cb5b55cbeef'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:04.647031 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590f50a44aa540d181aa8cb5b55cbeef'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:04.752620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1053/2113] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:11:04 2022] DELETE /identity/v3/users/590f50a44aa540d181aa8cb5b55cbeef => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:04.757687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcd360-c309-43b3-a501-4f0377d86de0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:04.758270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcd360-c309-43b3-a501-4f0377d86de0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:04.758366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcd360-c309-43b3-a501-4f0377d86de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:04.784946 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8fbcd360-c309-43b3-a501-4f0377d86de0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:04.786152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8fbcd360-c309-43b3-a501-4f0377d86de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:04.847678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1061/2114] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:04.857387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:04.858780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:04.918119 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4ff7b-b7b3-4330-9ca2-d29bda424985 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:04.921920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc88d1a-cf46-4ec6-87f2-6469d9048f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:04.922568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc88d1a-cf46-4ec6-87f2-6469d9048f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:04.922851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc88d1a-cf46-4ec6-87f2-6469d9048f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:04.923486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc88d1a-cf46-4ec6-87f2-6469d9048f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:04.934998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc88d1a-cf46-4ec6-87f2-6469d9048f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:04.936421 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc88d1a-cf46-4ec6-87f2-6469d9048f25 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:04.947144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1054/2115] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:05.387844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcd360-c309-43b3-a501-4f0377d86de0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:05.392046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8fbcd360-c309-43b3-a501-4f0377d86de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:05.402047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcd360-c309-43b3-a501-4f0377d86de0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:05.409248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:05.413682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:05.413961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:05.414240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] PATH_INFO: `/v3/projects/6f629484a82d4b8b82827acb08c4be5f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:05.418896 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f629484a82d4b8b82827acb08c4be5f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:05.420886 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:05.512158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f629484a82d4b8b82827acb08c4be5f', 'request_id': 'req-98821bf7-44fc-465f-a8ad-b075d58fde47'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:05.517682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f629484a82d4b8b82827acb08c4be5f', 'request_id': 'req-98821bf7-44fc-465f-a8ad-b075d58fde47'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:05.523813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f629484a82d4b8b82827acb08c4be5f', 'request_id': 'req-98821bf7-44fc-465f-a8ad-b075d58fde47'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:05.596388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] The token cache is being invalidate because project 6f629484a82d4b8b82827acb08c4be5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:11:05.596674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:05.597346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:05.600650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1062/2116] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:11:05 2022] DELETE /identity/v3/projects/6f629484a82d4b8b82827acb08c4be5f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:05.603544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc88d1a-cf46-4ec6-87f2-6469d9048f25 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:05.613497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc88d1a-cf46-4ec6-87f2-6469d9048f25 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:05.631256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:05.669145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:05.670002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:05.670547 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:05.670839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] PATH_INFO: `/v3/users/cd8eeb28a8a348ac8ef5605c111265ef` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:05.672036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd8eeb28a8a348ac8ef5605c111265ef)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:05.673979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:05.877123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd8eeb28a8a348ac8ef5605c111265ef'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:05.881263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd8eeb28a8a348ac8ef5605c111265ef'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:05.881925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd8eeb28a8a348ac8ef5605c111265ef'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:05.896097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd8eeb28a8a348ac8ef5605c111265ef'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:05.896782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6ff31db9-d855-4955-b33b-81de0b6cbf4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd8eeb28a8a348ac8ef5605c111265ef'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:05.971966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1055/2117] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:11:05 2022] DELETE /identity/v3/users/cd8eeb28a8a348ac8ef5605c111265ef => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:06.442033 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:06.443482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:06.456912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98821bf7-44fc-465f-a8ad-b075d58fde47 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:06.522439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:06.523299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:06.523579 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:06.523854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] PATH_INFO: `/v3/projects/f067e36406b1473ebc3057f8b1c9cd7f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:06.525640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f067e36406b1473ebc3057f8b1c9cd7f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:06.527588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:06.624732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:06.640837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f067e36406b1473ebc3057f8b1c9cd7f', 'request_id': 'req-0f44b39a-4ce1-403d-8e19-3da55af7d73a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:06.646372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f067e36406b1473ebc3057f8b1c9cd7f', 'request_id': 'req-0f44b39a-4ce1-403d-8e19-3da55af7d73a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:06.653221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f067e36406b1473ebc3057f8b1c9cd7f', 'request_id': 'req-0f44b39a-4ce1-403d-8e19-3da55af7d73a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:06.754598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] The token cache is being invalidate because project f067e36406b1473ebc3057f8b1c9cd7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:11:06.754885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:06.755623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0f44b39a-4ce1-403d-8e19-3da55af7d73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:06.757190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1063/2118] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:11:06 2022] DELETE /identity/v3/projects/f067e36406b1473ebc3057f8b1c9cd7f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:06.764227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e1e47-b78e-414d-b5cd-7d8a93b99e07 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:06.764815 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e1e47-b78e-414d-b5cd-7d8a93b99e07 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:06.765433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e1e47-b78e-414d-b5cd-7d8a93b99e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:06.783110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-2b4e1e47-b78e-414d-b5cd-7d8a93b99e07 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:06.784319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2b4e1e47-b78e-414d-b5cd-7d8a93b99e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:06.858154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1056/2119] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:06.865884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b05ff-9e29-4104-a5d3-2598ddde24e1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:06.866176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b05ff-9e29-4104-a5d3-2598ddde24e1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:06.866481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b05ff-9e29-4104-a5d3-2598ddde24e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:06.886155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e63b05ff-9e29-4104-a5d3-2598ddde24e1 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:06.887470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e63b05ff-9e29-4104-a5d3-2598ddde24e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:06.895769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1064/2120] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:06.900025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e1e47-b78e-414d-b5cd-7d8a93b99e07 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:06.909706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e1e47-b78e-414d-b5cd-7d8a93b99e07 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:06.914214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cefa16-c5a4-4bea-b15a-10475ac88875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:06.914852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cefa16-c5a4-4bea-b15a-10475ac88875 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:06.915135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cefa16-c5a4-4bea-b15a-10475ac88875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:06.915410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cefa16-c5a4-4bea-b15a-10475ac88875 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:06.916136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cefa16-c5a4-4bea-b15a-10475ac88875 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:11:06.916522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cefa16-c5a4-4bea-b15a-10475ac88875 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:06.918171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cefa16-c5a4-4bea-b15a-10475ac88875 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:06.929659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1057/2121] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:11:06.934462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b05ff-9e29-4104-a5d3-2598ddde24e1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:06.944256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b05ff-9e29-4104-a5d3-2598ddde24e1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:06.949726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bb144-0591-4be1-a48c-8f8f9da6c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:06.950396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bb144-0591-4be1-a48c-8f8f9da6c507 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:06.950675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bb144-0591-4be1-a48c-8f8f9da6c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:06.950949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9bb144-0591-4be1-a48c-8f8f9da6c507 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:06.951691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9bb144-0591-4be1-a48c-8f8f9da6c507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:06.953741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9bb144-0591-4be1-a48c-8f8f9da6c507 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:07.031098 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1065/2122] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:11:06 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:11:07.035187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cefa16-c5a4-4bea-b15a-10475ac88875 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:07.049202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cefa16-c5a4-4bea-b15a-10475ac88875 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:07.085132 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:07.118613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056feed-d615-41d6-9b3e-531e029a6696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:07.118889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056feed-d615-41d6-9b3e-531e029a6696 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:07.119183 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056feed-d615-41d6-9b3e-531e029a6696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:07.119465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056feed-d615-41d6-9b3e-531e029a6696 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:07.120163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056feed-d615-41d6-9b3e-531e029a6696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:07.122283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056feed-d615-41d6-9b3e-531e029a6696 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:07.259417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1058/2123] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:11:07 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:11:07.259590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bb144-0591-4be1-a48c-8f8f9da6c507 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:07.269645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9bb144-0591-4be1-a48c-8f8f9da6c507 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:07.274349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e3ee0-7938-472d-b8cc-d1319d8c2208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:07.275022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e3ee0-7938-472d-b8cc-d1319d8c2208 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:07.275287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e3ee0-7938-472d-b8cc-d1319d8c2208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:07.275561 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e3ee0-7938-472d-b8cc-d1319d8c2208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:07.278530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e3ee0-7938-472d-b8cc-d1319d8c2208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:07.280187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e3ee0-7938-472d-b8cc-d1319d8c2208 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:07.291452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1066/2124] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:11:07.295494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056feed-d615-41d6-9b3e-531e029a6696 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:07.306083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056feed-d615-41d6-9b3e-531e029a6696 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:07.310885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3759f-9e98-42f1-8060-c92b0413a535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:07.311532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3759f-9e98-42f1-8060-c92b0413a535 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:07.311816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3759f-9e98-42f1-8060-c92b0413a535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:07.312096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3759f-9e98-42f1-8060-c92b0413a535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:07.312853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3759f-9e98-42f1-8060-c92b0413a535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:07.314424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3759f-9e98-42f1-8060-c92b0413a535 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:07.324427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1059/2125] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:11:07.328533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e3ee0-7938-472d-b8cc-d1319d8c2208 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:07.338532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e3ee0-7938-472d-b8cc-d1319d8c2208 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:07.347066 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e517456-cd18-4519-8b18-e7bda8099cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:07.347066 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517456-cd18-4519-8b18-e7bda8099cf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:07.347066 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517456-cd18-4519-8b18-e7bda8099cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:07.347066 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517456-cd18-4519-8b18-e7bda8099cf5 admin admin] PATH_INFO: `/v3/projects/cec40e255eaa44f8951e0435b1166991/users/d5fd4d261f9b49c68eb3471d147b0ce8/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:07.361241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e517456-cd18-4519-8b18-e7bda8099cf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec40e255eaa44f8951e0435b1166991, user_id=d5fd4d261f9b49c68eb3471d147b0ce8, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:07.364231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e517456-cd18-4519-8b18-e7bda8099cf5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:07.460577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1067/2126] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:11:07 2022] PUT /identity/v3/projects/cec40e255eaa44f8951e0435b1166991/users/d5fd4d261f9b49c68eb3471d147b0ce8/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:07.479323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62820287-cebb-4b0c-b7e5-0da8d1fb86b9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:07.479323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62820287-cebb-4b0c-b7e5-0da8d1fb86b9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:07.479323 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62820287-cebb-4b0c-b7e5-0da8d1fb86b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:07.536494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-62820287-cebb-4b0c-b7e5-0da8d1fb86b9 None None] MFA Rules not processed for user `d5fd4d261f9b49c68eb3471d147b0ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:07.537738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-62820287-cebb-4b0c-b7e5-0da8d1fb86b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:07.608058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1060/2127] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:07.614338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e517456-cd18-4519-8b18-e7bda8099cf5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:07.697989 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e517456-cd18-4519-8b18-e7bda8099cf5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:07.702742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1b019-1ad0-4154-afc3-681d0557808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:07.703475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1b019-1ad0-4154-afc3-681d0557808e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:07.703760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1b019-1ad0-4154-afc3-681d0557808e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:07.704032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1b019-1ad0-4154-afc3-681d0557808e admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:07.704714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1b019-1ad0-4154-afc3-681d0557808e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:07.706668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1b019-1ad0-4154-afc3-681d0557808e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:07.871767 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1068/2128] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:11:07 2022] POST /identity/v3/projects => generated 379 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:11:07.875981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62820287-cebb-4b0c-b7e5-0da8d1fb86b9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:07.887465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62820287-cebb-4b0c-b7e5-0da8d1fb86b9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:07.954901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad963e7-63fc-4a8d-a861-80216f2b0ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:07.955771 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad963e7-63fc-4a8d-a861-80216f2b0ada admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:07.956109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad963e7-63fc-4a8d-a861-80216f2b0ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:07.956446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad963e7-63fc-4a8d-a861-80216f2b0ada admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:07.957272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad963e7-63fc-4a8d-a861-80216f2b0ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:07.959389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad963e7-63fc-4a8d-a861-80216f2b0ada admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:08.074140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1061/2129] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:11:07 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:11:08.075416 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1b019-1ad0-4154-afc3-681d0557808e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:08.086452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1b019-1ad0-4154-afc3-681d0557808e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:08.091813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef992807-12e3-4604-875b-3501a1ce5711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:08.092479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef992807-12e3-4604-875b-3501a1ce5711 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:08.092810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef992807-12e3-4604-875b-3501a1ce5711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:08.093091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef992807-12e3-4604-875b-3501a1ce5711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:08.093819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef992807-12e3-4604-875b-3501a1ce5711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:08.095433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef992807-12e3-4604-875b-3501a1ce5711 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:08.109924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1069/2130] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:11:08.114362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad963e7-63fc-4a8d-a861-80216f2b0ada admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:08.125412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad963e7-63fc-4a8d-a861-80216f2b0ada admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:08.130375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f60189-171f-4303-8fa8-eca4e1f37262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:08.131256 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f60189-171f-4303-8fa8-eca4e1f37262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:08.131538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f60189-171f-4303-8fa8-eca4e1f37262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:08.131817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f60189-171f-4303-8fa8-eca4e1f37262 admin admin] PATH_INFO: `/v3/projects/0104fc3967eb410bbbdce215aa7bad3e/users/8cda42a8edc942f7a6914bcf7bd1e29d/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:08.148473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f60189-171f-4303-8fa8-eca4e1f37262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0104fc3967eb410bbbdce215aa7bad3e, user_id=8cda42a8edc942f7a6914bcf7bd1e29d, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:08.151966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f60189-171f-4303-8fa8-eca4e1f37262 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:08.231765 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1062/2131] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:11:08 2022] PUT /identity/v3/projects/0104fc3967eb410bbbdce215aa7bad3e/users/8cda42a8edc942f7a6914bcf7bd1e29d/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:11:08.232729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef992807-12e3-4604-875b-3501a1ce5711 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:08.251692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef992807-12e3-4604-875b-3501a1ce5711 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:08.281454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:08.314206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f6cef-db23-4c44-a3a4-d2763a409d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:08.314902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f6cef-db23-4c44-a3a4-d2763a409d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:08.315188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f6cef-db23-4c44-a3a4-d2763a409d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:08.315467 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f6cef-db23-4c44-a3a4-d2763a409d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:08.316245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f6cef-db23-4c44-a3a4-d2763a409d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:08.317921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f6cef-db23-4c44-a3a4-d2763a409d2c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:08.328252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1070/2132] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:11:08.332593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f60189-171f-4303-8fa8-eca4e1f37262 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:08.340100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:08.346434 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f60189-171f-4303-8fa8-eca4e1f37262 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:08.351662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737ff53-5fca-4063-bd68-fddd7dd28b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:08.352596 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737ff53-5fca-4063-bd68-fddd7dd28b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:08.352919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737ff53-5fca-4063-bd68-fddd7dd28b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:08.353196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737ff53-5fca-4063-bd68-fddd7dd28b62 admin admin] PATH_INFO: `/v3/projects/0104fc3967eb410bbbdce215aa7bad3e/users/8cda42a8edc942f7a6914bcf7bd1e29d/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:08.355692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737ff53-5fca-4063-bd68-fddd7dd28b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0104fc3967eb410bbbdce215aa7bad3e, user_id=8cda42a8edc942f7a6914bcf7bd1e29d, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:08.360354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737ff53-5fca-4063-bd68-fddd7dd28b62 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:08.373216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1063/2133] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:11:08 2022] PUT /identity/v3/projects/0104fc3967eb410bbbdce215aa7bad3e/users/8cda42a8edc942f7a6914bcf7bd1e29d/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:08.384178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e25ea-c531-49bf-a927-bed569b4257d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:08.384640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e25ea-c531-49bf-a927-bed569b4257d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:08.384955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e25ea-c531-49bf-a927-bed569b4257d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:08.420759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c16e25ea-c531-49bf-a927-bed569b4257d None None] MFA Rules not processed for user `8cda42a8edc942f7a6914bcf7bd1e29d`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:08.422042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c16e25ea-c531-49bf-a927-bed569b4257d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:08.494447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1071/2134] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:08.506519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737ff53-5fca-4063-bd68-fddd7dd28b62 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:08.508186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3737ff53-5fca-4063-bd68-fddd7dd28b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:08.576577 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737ff53-5fca-4063-bd68-fddd7dd28b62 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:08.580314 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf1af1-9426-4d29-8399-9117da6c8d1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:08.580988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf1af1-9426-4d29-8399-9117da6c8d1a service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:08.581273 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf1af1-9426-4d29-8399-9117da6c8d1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:08.581555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf1af1-9426-4d29-8399-9117da6c8d1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:08.591029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf1af1-9426-4d29-8399-9117da6c8d1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:08.592479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf1af1-9426-4d29-8399-9117da6c8d1a service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:08.602544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1064/2135] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:09.240654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e25ea-c531-49bf-a927-bed569b4257d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:09.355076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e25ea-c531-49bf-a927-bed569b4257d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:09.360076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:09.360982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:09.361272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:09.361554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] PATH_INFO: `/v3/users/d5fd4d261f9b49c68eb3471d147b0ce8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:09.362816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5fd4d261f9b49c68eb3471d147b0ce8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:09.364863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:09.369711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:09.578970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5fd4d261f9b49c68eb3471d147b0ce8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:09.586984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5fd4d261f9b49c68eb3471d147b0ce8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:09.587731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5fd4d261f9b49c68eb3471d147b0ce8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:09.588321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5fd4d261f9b49c68eb3471d147b0ce8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:09.600209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5fd4d261f9b49c68eb3471d147b0ce8'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:09.702058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1072/2136] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:11:09 2022] DELETE /identity/v3/users/d5fd4d261f9b49c68eb3471d147b0ce8 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:09.706809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d86a7-c5ed-4225-ab12-b833a93814f1 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:09.707299 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d86a7-c5ed-4225-ab12-b833a93814f1 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:09.707582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d86a7-c5ed-4225-ab12-b833a93814f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:09.771243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-633d86a7-c5ed-4225-ab12-b833a93814f1 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:09.772521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-633d86a7-c5ed-4225-ab12-b833a93814f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:09.836886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1065/2137] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:09.848988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:09.852158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:09.966809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2e968-06f9-4512-bf25-53d71392d5d9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:09.970659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d4e45-b00f-4d66-9b7d-4b1087304338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:09.971309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d4e45-b00f-4d66-9b7d-4b1087304338 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:09.971759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d4e45-b00f-4d66-9b7d-4b1087304338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:09.971847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d4e45-b00f-4d66-9b7d-4b1087304338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:09.999070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d4e45-b00f-4d66-9b7d-4b1087304338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:10.000689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d4e45-b00f-4d66-9b7d-4b1087304338 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:10.012975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1073/2138] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:10.062837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3cd3c-0ba4-4749-b219-8155aab5d401 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:10.063337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3cd3c-0ba4-4749-b219-8155aab5d401 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:10.063692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3cd3c-0ba4-4749-b219-8155aab5d401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:10.085671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b5a3cd3c-0ba4-4749-b219-8155aab5d401 None None] MFA Rules not processed for user `8b6e665914774eabba120e986e6bb282`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:10.086929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b5a3cd3c-0ba4-4749-b219-8155aab5d401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:10.149302 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1066/2139] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:10.507088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d4e45-b00f-4d66-9b7d-4b1087304338 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:10.507088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e83d4e45-b00f-4d66-9b7d-4b1087304338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:10.532998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d4e45-b00f-4d66-9b7d-4b1087304338 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:10.548601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:10.549543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:10.549828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:10.550140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] PATH_INFO: `/v3/projects/cec40e255eaa44f8951e0435b1166991` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:10.553082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec40e255eaa44f8951e0435b1166991)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:10.555074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:10.745736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:10.749232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec40e255eaa44f8951e0435b1166991', 'request_id': 'req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:10.755750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec40e255eaa44f8951e0435b1166991', 'request_id': 'req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:10.763931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec40e255eaa44f8951e0435b1166991', 'request_id': 'req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:10.892644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] The token cache is being invalidate because project cec40e255eaa44f8951e0435b1166991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:11:10.895868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:10.895868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:10.895868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1074/2140] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:11:10 2022] DELETE /identity/v3/projects/cec40e255eaa44f8951e0435b1166991 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:10.900078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3cd3c-0ba4-4749-b219-8155aab5d401 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:10.910920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3cd3c-0ba4-4749-b219-8155aab5d401 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:10.920163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:10.981009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:10.982407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:10.982687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:10.986606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] PATH_INFO: `/v3/users/8cda42a8edc942f7a6914bcf7bd1e29d` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:10.986606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cda42a8edc942f7a6914bcf7bd1e29d)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:10.987035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:11.322405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cda42a8edc942f7a6914bcf7bd1e29d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:11.330688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cda42a8edc942f7a6914bcf7bd1e29d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:11.334192 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cda42a8edc942f7a6914bcf7bd1e29d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:11.353037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cda42a8edc942f7a6914bcf7bd1e29d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:11.357197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-2799156d-7a74-4f21-a082-8c465946747b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cda42a8edc942f7a6914bcf7bd1e29d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:11.483427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1067/2141] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:11:10 2022] DELETE /identity/v3/users/8cda42a8edc942f7a6914bcf7bd1e29d => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:11:12.062147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:12.063687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:12.080798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeef7f0-4f4d-4f5f-89e0-9f8f068614b1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:12.151388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:12.152558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:12.152964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:12.153472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] PATH_INFO: `/v3/projects/0104fc3967eb410bbbdce215aa7bad3e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:12.154878 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0104fc3967eb410bbbdce215aa7bad3e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:12.157023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:12.186507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e7a55-7f46-4238-81ec-af974a103731 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:12.186507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e7a55-7f46-4238-81ec-af974a103731 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:12.186507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e7a55-7f46-4238-81ec-af974a103731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:12.203116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-6a9e7a55-7f46-4238-81ec-af974a103731 None None] MFA Rules not processed for user `0c8512e7c1704fac9cc4fcfe3bf1f18b`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:12.204509 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-6a9e7a55-7f46-4238-81ec-af974a103731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:12.263097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1068/2142] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:12.385925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0104fc3967eb410bbbdce215aa7bad3e', 'request_id': 'req-c2730388-b0c7-401f-bf79-cc3d5fe64da9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:12.391814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0104fc3967eb410bbbdce215aa7bad3e', 'request_id': 'req-c2730388-b0c7-401f-bf79-cc3d5fe64da9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:12.398581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0104fc3967eb410bbbdce215aa7bad3e', 'request_id': 'req-c2730388-b0c7-401f-bf79-cc3d5fe64da9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:12.474407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] The token cache is being invalidate because project 0104fc3967eb410bbbdce215aa7bad3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:11:12.475019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:12.475836 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c2730388-b0c7-401f-bf79-cc3d5fe64da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:12.478530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1075/2143] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:11:12 2022] DELETE /identity/v3/projects/0104fc3967eb410bbbdce215aa7bad3e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:18.407692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e7a55-7f46-4238-81ec-af974a103731 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:18.409167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-6a9e7a55-7f46-4238-81ec-af974a103731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:18.435295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e7a55-7f46-4238-81ec-af974a103731 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:18.516356 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:18.557338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:18.558482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:18.558865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:18.559377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] PATH_INFO: `/v3/users/8311a61e39b64f00b9235973f7414cfe` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:18.561152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8311a61e39b64f00b9235973f7414cfe)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:18.576736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:18.779465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8311a61e39b64f00b9235973f7414cfe'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:18.796331 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8311a61e39b64f00b9235973f7414cfe'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:18.797110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8311a61e39b64f00b9235973f7414cfe'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:18.809692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8311a61e39b64f00b9235973f7414cfe'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:18.810426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8311a61e39b64f00b9235973f7414cfe'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:18.933187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1069/2144] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:11:18 2022] DELETE /identity/v3/users/8311a61e39b64f00b9235973f7414cfe => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:18.938090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ee6a3-4ee6-4a4f-a096-b0162c233cda None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:18.938609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ee6a3-4ee6-4a4f-a096-b0162c233cda None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:18.938753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ee6a3-4ee6-4a4f-a096-b0162c233cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:18.956472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2b3ee6a3-4ee6-4a4f-a096-b0162c233cda None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:18.958255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2b3ee6a3-4ee6-4a4f-a096-b0162c233cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:19.026617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1076/2145] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:19.050659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:19.052748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:19.134488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce590b9-39e8-4379-afdc-fa6253575b0a admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:19.138458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc7f77-9abc-44ea-b887-b13fe8cdccce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:19.139128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc7f77-9abc-44ea-b887-b13fe8cdccce service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:19.143317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc7f77-9abc-44ea-b887-b13fe8cdccce service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:19.143684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc7f77-9abc-44ea-b887-b13fe8cdccce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:19.165699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc7f77-9abc-44ea-b887-b13fe8cdccce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:19.167933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc7f77-9abc-44ea-b887-b13fe8cdccce service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:19.193696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1070/2146] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:19.719384 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ee6a3-4ee6-4a4f-a096-b0162c233cda None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:19.727930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2b3ee6a3-4ee6-4a4f-a096-b0162c233cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:19.742763 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:19.747237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ee6a3-4ee6-4a4f-a096-b0162c233cda None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:19.752181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:19.753197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:19.753538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:19.757781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] PATH_INFO: `/v3/projects/532ea5c6fc364dc79591e551e2d3b8eb` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:19.757781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=532ea5c6fc364dc79591e551e2d3b8eb)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:19.757781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:19.925619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532ea5c6fc364dc79591e551e2d3b8eb', 'request_id': 'req-9c14abe0-d893-4816-9ae5-177760bbe441'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:19.938357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532ea5c6fc364dc79591e551e2d3b8eb', 'request_id': 'req-9c14abe0-d893-4816-9ae5-177760bbe441'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:19.948725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532ea5c6fc364dc79591e551e2d3b8eb', 'request_id': 'req-9c14abe0-d893-4816-9ae5-177760bbe441'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:20.033770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] The token cache is being invalidate because project 532ea5c6fc364dc79591e551e2d3b8eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:11:20.035211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:20.036524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-9c14abe0-d893-4816-9ae5-177760bbe441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:20.039498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1077/2147] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:11:19 2022] DELETE /identity/v3/projects/532ea5c6fc364dc79591e551e2d3b8eb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:20.052219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ad95c-1471-4774-b958-c77bf8849692 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:20.052737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ad95c-1471-4774-b958-c77bf8849692 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:20.052737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ad95c-1471-4774-b958-c77bf8849692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:20.102360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-874ad95c-1471-4774-b958-c77bf8849692 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:20.103614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-874ad95c-1471-4774-b958-c77bf8849692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:20.144666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:20.171999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1071/2148] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:20.180432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56414d60-6a4f-463e-857e-e372d8826f57 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:20.186549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56414d60-6a4f-463e-857e-e372d8826f57 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:20.186549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56414d60-6a4f-463e-857e-e372d8826f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:20.199298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-56414d60-6a4f-463e-857e-e372d8826f57 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:20.200522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-56414d60-6a4f-463e-857e-e372d8826f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:20.211882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1078/2149] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:20.215959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ad95c-1471-4774-b958-c77bf8849692 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:20.226229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ad95c-1471-4774-b958-c77bf8849692 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:20.231670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b5f96-8d25-4c41-8eab-fb60af713682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:20.232371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b5f96-8d25-4c41-8eab-fb60af713682 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:20.232666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b5f96-8d25-4c41-8eab-fb60af713682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:20.232983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b5f96-8d25-4c41-8eab-fb60af713682 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:20.233724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b5f96-8d25-4c41-8eab-fb60af713682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:11:20.234032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b5f96-8d25-4c41-8eab-fb60af713682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:20.235904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b5f96-8d25-4c41-8eab-fb60af713682 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:20.252148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1072/2150] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:11:20.253346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56414d60-6a4f-463e-857e-e372d8826f57 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:20.263711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56414d60-6a4f-463e-857e-e372d8826f57 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:20.268931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672be49-acb7-4417-a249-5d1c2e1966c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:20.269572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672be49-acb7-4417-a249-5d1c2e1966c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:20.269859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672be49-acb7-4417-a249-5d1c2e1966c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:20.270139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672be49-acb7-4417-a249-5d1c2e1966c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:20.270837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672be49-acb7-4417-a249-5d1c2e1966c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:20.272757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672be49-acb7-4417-a249-5d1c2e1966c0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:20.430084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1079/2151] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:11:20 2022] POST /identity/v3/projects => generated 397 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:11:20.435456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b5f96-8d25-4c41-8eab-fb60af713682 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:20.446272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b5f96-8d25-4c41-8eab-fb60af713682 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:20.506346 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a091e32-122b-4329-b4bd-043f380617ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:20.507019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a091e32-122b-4329-b4bd-043f380617ef admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:20.509673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a091e32-122b-4329-b4bd-043f380617ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:20.509673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a091e32-122b-4329-b4bd-043f380617ef admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:20.509673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a091e32-122b-4329-b4bd-043f380617ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:20.510248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a091e32-122b-4329-b4bd-043f380617ef admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:20.631008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1073/2152] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:11:20 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:11:20.631108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672be49-acb7-4417-a249-5d1c2e1966c0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:20.647184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672be49-acb7-4417-a249-5d1c2e1966c0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:20.663091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cc46e-e21a-41a7-a4df-b56d166ab3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:20.664001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cc46e-e21a-41a7-a4df-b56d166ab3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:20.664290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cc46e-e21a-41a7-a4df-b56d166ab3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:20.664569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cc46e-e21a-41a7-a4df-b56d166ab3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:20.665342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021cc46e-e21a-41a7-a4df-b56d166ab3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:20.666975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021cc46e-e21a-41a7-a4df-b56d166ab3f8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:20.691771 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1080/2153] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:11:20.696685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a091e32-122b-4329-b4bd-043f380617ef admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:20.707552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a091e32-122b-4329-b4bd-043f380617ef admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:20.714013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3319f34-aea8-44b4-a207-1cc93a24d28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:20.716944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3319f34-aea8-44b4-a207-1cc93a24d28d admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:20.716944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3319f34-aea8-44b4-a207-1cc93a24d28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:20.716944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3319f34-aea8-44b4-a207-1cc93a24d28d admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:20.716944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3319f34-aea8-44b4-a207-1cc93a24d28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:20.718582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3319f34-aea8-44b4-a207-1cc93a24d28d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:20.738761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1074/2154] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:11:20.741057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cc46e-e21a-41a7-a4df-b56d166ab3f8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:20.765427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cc46e-e21a-41a7-a4df-b56d166ab3f8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:20.772278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1e8c0-c825-48e8-9fa1-41f836f756cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:20.774336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1e8c0-c825-48e8-9fa1-41f836f756cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:20.776415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1e8c0-c825-48e8-9fa1-41f836f756cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:20.776938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1e8c0-c825-48e8-9fa1-41f836f756cb admin admin] PATH_INFO: `/v3/projects/9bca3df3dbce4ee69c7081d663cddc4b/users/c25cc3e3e88f49a2a36ddbeae857ebce/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:20.802350 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1e8c0-c825-48e8-9fa1-41f836f756cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bca3df3dbce4ee69c7081d663cddc4b, user_id=c25cc3e3e88f49a2a36ddbeae857ebce, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:20.807021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1e8c0-c825-48e8-9fa1-41f836f756cb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:20.875973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1081/2155] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:11:20 2022] PUT /identity/v3/projects/9bca3df3dbce4ee69c7081d663cddc4b/users/c25cc3e3e88f49a2a36ddbeae857ebce/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:20.960072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa4f75-1dbd-4590-9d91-6609851623b0 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:20.960500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa4f75-1dbd-4590-9d91-6609851623b0 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:20.960964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa4f75-1dbd-4590-9d91-6609851623b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:21.029467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-06aa4f75-1dbd-4590-9d91-6609851623b0 None None] MFA Rules not processed for user `c25cc3e3e88f49a2a36ddbeae857ebce`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:21.030939 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-06aa4f75-1dbd-4590-9d91-6609851623b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:21.128239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1075/2156] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:21.128933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1e8c0-c825-48e8-9fa1-41f836f756cb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:21.130544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7dc1e8c0-c825-48e8-9fa1-41f836f756cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:21.138641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa4f75-1dbd-4590-9d91-6609851623b0 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:21.139895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-06aa4f75-1dbd-4590-9d91-6609851623b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:21.203404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:21.221754 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa4f75-1dbd-4590-9d91-6609851623b0 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:21.222842 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1e8c0-c825-48e8-9fa1-41f836f756cb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:21.226086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2ffb8-57e3-49f2-9fe5-fbea094de9c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:21.226924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ffb8-57e3-49f2-9fe5-fbea094de9c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:21.227264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ffb8-57e3-49f2-9fe5-fbea094de9c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:21.227663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2ffb8-57e3-49f2-9fe5-fbea094de9c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:21.228106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:21.229132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:21.229417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:21.229746 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] PATH_INFO: `/v3/users/8dee1a69a2ec4043b17a59613b1e8f83` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:21.230969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dee1a69a2ec4043b17a59613b1e8f83)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:21.233149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:21.237833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2ffb8-57e3-49f2-9fe5-fbea094de9c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:21.239269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2ffb8-57e3-49f2-9fe5-fbea094de9c5 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:21.250362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1076/2157] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:11:21.375784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:21.493647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dee1a69a2ec4043b17a59613b1e8f83'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:21.501613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dee1a69a2ec4043b17a59613b1e8f83'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:21.502670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dee1a69a2ec4043b17a59613b1e8f83'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:21.503283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dee1a69a2ec4043b17a59613b1e8f83'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:21.519988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dee1a69a2ec4043b17a59613b1e8f83'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:21.650268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1082/2158] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:11:21 2022] DELETE /identity/v3/users/8dee1a69a2ec4043b17a59613b1e8f83 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:11:21.655131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982fe10-6ba9-49c8-b4a9-472ba24e7d75 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:21.655424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982fe10-6ba9-49c8-b4a9-472ba24e7d75 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:21.655706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982fe10-6ba9-49c8-b4a9-472ba24e7d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:21.673974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-9982fe10-6ba9-49c8-b4a9-472ba24e7d75 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:21.675405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9982fe10-6ba9-49c8-b4a9-472ba24e7d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:21.741041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1077/2159] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:21.753171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:21.754731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:21.831745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ed79e-577c-4d81-8abb-e37dbdc7602d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:21.835730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134bb7a-9e36-451b-a279-731a13e5ba46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:21.838462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134bb7a-9e36-451b-a279-731a13e5ba46 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:21.838462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134bb7a-9e36-451b-a279-731a13e5ba46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:21.838462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134bb7a-9e36-451b-a279-731a13e5ba46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:21.847469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134bb7a-9e36-451b-a279-731a13e5ba46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:21.848952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134bb7a-9e36-451b-a279-731a13e5ba46 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:21.863478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1083/2160] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:22.322155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982fe10-6ba9-49c8-b4a9-472ba24e7d75 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:22.324192 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9982fe10-6ba9-49c8-b4a9-472ba24e7d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:22.340865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982fe10-6ba9-49c8-b4a9-472ba24e7d75 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:22.347987 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:22.349737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:22.350178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:22.351272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] PATH_INFO: `/v3/projects/e98dfd576b4d465095388846eff5da5f` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:22.352799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e98dfd576b4d465095388846eff5da5f)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:22.355239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:22.513475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e98dfd576b4d465095388846eff5da5f', 'request_id': 'req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:22.530101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e98dfd576b4d465095388846eff5da5f', 'request_id': 'req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:22.583511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e98dfd576b4d465095388846eff5da5f', 'request_id': 'req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:22.599309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] The token cache is being invalidate because project e98dfd576b4d465095388846eff5da5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:11:22.600281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:22.602168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6f1f8ee8-59a1-4fcf-9b17-4b7b9775fe49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:22.605040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1078/2161] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:11:22 2022] DELETE /identity/v3/projects/e98dfd576b4d465095388846eff5da5f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:22.610880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab81fd-c47a-47c7-bdf9-fd0a539d28f0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:22.611330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab81fd-c47a-47c7-bdf9-fd0a539d28f0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:22.611741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab81fd-c47a-47c7-bdf9-fd0a539d28f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:22.655382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3aab81fd-c47a-47c7-bdf9-fd0a539d28f0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:22.656497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3aab81fd-c47a-47c7-bdf9-fd0a539d28f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:22.796587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1084/2162] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:22.811713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e15984-7ac7-423a-878c-10e92e31b2de None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:22.812010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e15984-7ac7-423a-878c-10e92e31b2de None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:22.812340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e15984-7ac7-423a-878c-10e92e31b2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:22.832615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f7e15984-7ac7-423a-878c-10e92e31b2de None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:22.833963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f7e15984-7ac7-423a-878c-10e92e31b2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:22.842964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1079/2163] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:11:22.847529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab81fd-c47a-47c7-bdf9-fd0a539d28f0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:22.863274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab81fd-c47a-47c7-bdf9-fd0a539d28f0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:22.875774 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609138a-6ee8-47c6-87cd-af79ea3b01f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:22.876741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609138a-6ee8-47c6-87cd-af79ea3b01f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:22.877259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609138a-6ee8-47c6-87cd-af79ea3b01f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:22.877912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609138a-6ee8-47c6-87cd-af79ea3b01f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:22.878729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609138a-6ee8-47c6-87cd-af79ea3b01f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:11:22.879041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609138a-6ee8-47c6-87cd-af79ea3b01f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:22.881229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609138a-6ee8-47c6-87cd-af79ea3b01f6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:22.905470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1085/2164] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:11:22.909705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e15984-7ac7-423a-878c-10e92e31b2de None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:22.925422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e15984-7ac7-423a-878c-10e92e31b2de None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:22.934956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492651a-c9c6-4b28-ac90-31c0e40b4e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:22.935202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492651a-c9c6-4b28-ac90-31c0e40b4e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:22.935540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492651a-c9c6-4b28-ac90-31c0e40b4e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:22.935877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492651a-c9c6-4b28-ac90-31c0e40b4e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:22.937063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492651a-c9c6-4b28-ac90-31c0e40b4e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:22.939425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492651a-c9c6-4b28-ac90-31c0e40b4e93 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:23.067207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1080/2165] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:11:22 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:11:23.072662 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609138a-6ee8-47c6-87cd-af79ea3b01f6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:23.087040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609138a-6ee8-47c6-87cd-af79ea3b01f6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:23.114800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:23.164575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf5f8d-94d3-4f6c-9e9a-251a0dcb7de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:23.165334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf5f8d-94d3-4f6c-9e9a-251a0dcb7de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:23.165668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf5f8d-94d3-4f6c-9e9a-251a0dcb7de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:23.165967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf5f8d-94d3-4f6c-9e9a-251a0dcb7de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:23.166835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf5f8d-94d3-4f6c-9e9a-251a0dcb7de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:23.169455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf5f8d-94d3-4f6c-9e9a-251a0dcb7de0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:23.266685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1086/2166] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:11:23 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:11:23.273778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492651a-c9c6-4b28-ac90-31c0e40b4e93 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:23.300796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492651a-c9c6-4b28-ac90-31c0e40b4e93 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:23.327350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c0096-3b05-4fe1-9800-2291136f0dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:23.332083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c0096-3b05-4fe1-9800-2291136f0dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:23.332083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c0096-3b05-4fe1-9800-2291136f0dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:23.332083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c0096-3b05-4fe1-9800-2291136f0dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:23.332083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c0096-3b05-4fe1-9800-2291136f0dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:23.333315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c0096-3b05-4fe1-9800-2291136f0dd4 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:23.352143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1081/2167] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:11:23.358042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf5f8d-94d3-4f6c-9e9a-251a0dcb7de0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:23.384638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf5f8d-94d3-4f6c-9e9a-251a0dcb7de0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:23.390355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1600e-1aa8-4f73-a8b3-5c1cd4934dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:23.391265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1600e-1aa8-4f73-a8b3-5c1cd4934dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:23.391740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1600e-1aa8-4f73-a8b3-5c1cd4934dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:23.392225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1600e-1aa8-4f73-a8b3-5c1cd4934dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:23.393190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1600e-1aa8-4f73-a8b3-5c1cd4934dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:23.395241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1600e-1aa8-4f73-a8b3-5c1cd4934dd5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:23.407162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1087/2168] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:11:23.411639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c0096-3b05-4fe1-9800-2291136f0dd4 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:23.422865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c0096-3b05-4fe1-9800-2291136f0dd4 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:23.427870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba2df-c596-436a-8e96-9d6901bdaca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:23.428912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba2df-c596-436a-8e96-9d6901bdaca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:23.429432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba2df-c596-436a-8e96-9d6901bdaca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:23.429726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba2df-c596-436a-8e96-9d6901bdaca6 admin admin] PATH_INFO: `/v3/projects/dce6460ca0714dc98df6e69034242a7f/users/6e0cc3e3c4804237b78241cca28ca689/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:23.445327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ba2df-c596-436a-8e96-9d6901bdaca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce6460ca0714dc98df6e69034242a7f, user_id=6e0cc3e3c4804237b78241cca28ca689, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:23.448047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ba2df-c596-436a-8e96-9d6901bdaca6 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:23.526479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:23.531734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1082/2169] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:11:23 2022] PUT /identity/v3/projects/dce6460ca0714dc98df6e69034242a7f/users/6e0cc3e3c4804237b78241cca28ca689/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:11:23.537561 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa339b-f995-425d-86f7-a3af7640fac1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:23.537859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa339b-f995-425d-86f7-a3af7640fac1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:23.538137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa339b-f995-425d-86f7-a3af7640fac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:23.557268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-63aa339b-f995-425d-86f7-a3af7640fac1 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:23.558615 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-63aa339b-f995-425d-86f7-a3af7640fac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:23.623090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1088/2170] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:23.633517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba2df-c596-436a-8e96-9d6901bdaca6 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:23.635476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8a1ba2df-c596-436a-8e96-9d6901bdaca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:23.707559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba2df-c596-436a-8e96-9d6901bdaca6 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:23.711865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff27ee0-5e7f-4f48-8532-ce41028573bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:23.712506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff27ee0-5e7f-4f48-8532-ce41028573bf service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:23.712823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff27ee0-5e7f-4f48-8532-ce41028573bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:23.713106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff27ee0-5e7f-4f48-8532-ce41028573bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:23.722906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff27ee0-5e7f-4f48-8532-ce41028573bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:23.724467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff27ee0-5e7f-4f48-8532-ce41028573bf service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:23.733976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1083/2171] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:25.063872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497548-0fd7-463f-90b5-21c0e9357ee9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:25.064420 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497548-0fd7-463f-90b5-21c0e9357ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:25.064499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497548-0fd7-463f-90b5-21c0e9357ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:25.096109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9f497548-0fd7-463f-90b5-21c0e9357ee9 None None] MFA Rules not processed for user `6e0cc3e3c4804237b78241cca28ca689`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:25.099899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9f497548-0fd7-463f-90b5-21c0e9357ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:25.179379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1089/2172] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:25.187859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff27ee0-5e7f-4f48-8532-ce41028573bf service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:25.190712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1ff27ee0-5e7f-4f48-8532-ce41028573bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:25.263820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff27ee0-5e7f-4f48-8532-ce41028573bf service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:25.275695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e118a4a-0eb8-4b05-a257-469b7f6ed3e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:25.275695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e118a4a-0eb8-4b05-a257-469b7f6ed3e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:25.275695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e118a4a-0eb8-4b05-a257-469b7f6ed3e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:25.275695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e118a4a-0eb8-4b05-a257-469b7f6ed3e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:25.289831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e118a4a-0eb8-4b05-a257-469b7f6ed3e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:25.292507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e118a4a-0eb8-4b05-a257-469b7f6ed3e5 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:25.302873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1084/2173] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:11:28.150785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ad0be-1318-4b48-bb1f-a820ddde5b30 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:28.150785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ad0be-1318-4b48-bb1f-a820ddde5b30 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:28.150785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ad0be-1318-4b48-bb1f-a820ddde5b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:28.167867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-375ad0be-1318-4b48-bb1f-a820ddde5b30 None None] MFA Rules not processed for user `6e0cc3e3c4804237b78241cca28ca689`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:28.169139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-375ad0be-1318-4b48-bb1f-a820ddde5b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:28.176198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1090/2174] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:30.572171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e118a4a-0eb8-4b05-a257-469b7f6ed3e5 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:30.573734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2e118a4a-0eb8-4b05-a257-469b7f6ed3e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:30.589530 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e118a4a-0eb8-4b05-a257-469b7f6ed3e5 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:30.594190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:30.595052 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:30.595332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:30.595607 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] PATH_INFO: `/v3/users/c25cc3e3e88f49a2a36ddbeae857ebce` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:30.596819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c25cc3e3e88f49a2a36ddbeae857ebce)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:30.598938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:30.849258 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c25cc3e3e88f49a2a36ddbeae857ebce'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:30.853922 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c25cc3e3e88f49a2a36ddbeae857ebce'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:30.856817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c25cc3e3e88f49a2a36ddbeae857ebce'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:30.883443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c25cc3e3e88f49a2a36ddbeae857ebce'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:30.884284 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c25cc3e3e88f49a2a36ddbeae857ebce'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:31.252006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1085/2175] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:11:30 2022] DELETE /identity/v3/users/c25cc3e3e88f49a2a36ddbeae857ebce => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:31.256647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b629f5-f4dd-4bae-823f-ca3302da5512 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:31.257198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b629f5-f4dd-4bae-823f-ca3302da5512 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:31.257301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b629f5-f4dd-4bae-823f-ca3302da5512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:31.275153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-66b629f5-f4dd-4bae-823f-ca3302da5512 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:31.276426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-66b629f5-f4dd-4bae-823f-ca3302da5512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:31.299875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:31.340053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1091/2176] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:31.350496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:31.354146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:31.383203 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:31.433050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd2524-2c73-4930-a722-57f3e66a564b admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:31.444800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301682c-af80-4a6a-9433-1e2e3acd33d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:31.445889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301682c-af80-4a6a-9433-1e2e3acd33d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:31.445889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301682c-af80-4a6a-9433-1e2e3acd33d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:31.445889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301682c-af80-4a6a-9433-1e2e3acd33d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:31.455678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301682c-af80-4a6a-9433-1e2e3acd33d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:31.457206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301682c-af80-4a6a-9433-1e2e3acd33d0 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:31.480755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1086/2177] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:32.011716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b629f5-f4dd-4bae-823f-ca3302da5512 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:32.014368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-66b629f5-f4dd-4bae-823f-ca3302da5512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:32.029052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b629f5-f4dd-4bae-823f-ca3302da5512 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:32.037946 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:32.038867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:32.039199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:32.039478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] PATH_INFO: `/v3/projects/9bca3df3dbce4ee69c7081d663cddc4b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:32.040695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bca3df3dbce4ee69c7081d663cddc4b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:32.042697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:32.152493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bca3df3dbce4ee69c7081d663cddc4b', 'request_id': 'req-3a46675e-8139-4042-a2de-392d49c2d340'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:32.158675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bca3df3dbce4ee69c7081d663cddc4b', 'request_id': 'req-3a46675e-8139-4042-a2de-392d49c2d340'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:32.165162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bca3df3dbce4ee69c7081d663cddc4b', 'request_id': 'req-3a46675e-8139-4042-a2de-392d49c2d340'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:32.240118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] The token cache is being invalidate because project 9bca3df3dbce4ee69c7081d663cddc4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:11:32.240407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:32.241053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:32.243090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1092/2178] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:11:32 2022] DELETE /identity/v3/projects/9bca3df3dbce4ee69c7081d663cddc4b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:32.924412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24eecb-6038-4938-b241-84870e74719d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:32.924412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24eecb-6038-4938-b241-84870e74719d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:32.924412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24eecb-6038-4938-b241-84870e74719d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:32.972843 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3e24eecb-6038-4938-b241-84870e74719d None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:32.974465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3e24eecb-6038-4938-b241-84870e74719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:33.101573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1087/2179] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:11:33.160556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:33.162592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:33.257660 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46675e-8139-4042-a2de-392d49c2d340 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:33.263007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c54f9f-abe2-4009-9574-3dcdbb2dab08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:33.263680 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c54f9f-abe2-4009-9574-3dcdbb2dab08 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:33.263969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c54f9f-abe2-4009-9574-3dcdbb2dab08 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:33.264248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c54f9f-abe2-4009-9574-3dcdbb2dab08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:33.278902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c54f9f-abe2-4009-9574-3dcdbb2dab08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:33.281393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c54f9f-abe2-4009-9574-3dcdbb2dab08 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:33.297643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1093/2180] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:11:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:11:43.092630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=50 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:43.288759 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:46.745478 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24eecb-6038-4938-b241-84870e74719d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:46.757661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3e24eecb-6038-4938-b241-84870e74719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:46.868496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24eecb-6038-4938-b241-84870e74719d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:46.883833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:46.884726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:46.885044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:46.885338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] PATH_INFO: `/v3/users/337ad1eb42c04f59b5b5e096b43c0c97` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:46.886760 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=337ad1eb42c04f59b5b5e096b43c0c97)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:46.889093 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:47.098696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '337ad1eb42c04f59b5b5e096b43c0c97'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:47.107458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '337ad1eb42c04f59b5b5e096b43c0c97'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:47.108195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '337ad1eb42c04f59b5b5e096b43c0c97'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:47.121594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '337ad1eb42c04f59b5b5e096b43c0c97'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:47.122307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '337ad1eb42c04f59b5b5e096b43c0c97'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:47.309111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1088/2181] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:11:46 2022] DELETE /identity/v3/users/337ad1eb42c04f59b5b5e096b43c0c97 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:47.313649 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2ad70-1172-4188-82c1-2447cdcfb577 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:47.314086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2ad70-1172-4188-82c1-2447cdcfb577 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:47.314198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2ad70-1172-4188-82c1-2447cdcfb577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:47.333943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-afc2ad70-1172-4188-82c1-2447cdcfb577 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:47.336895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-afc2ad70-1172-4188-82c1-2447cdcfb577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:47.402737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1094/2182] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:47.412629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:47.414040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:47.480066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042c79d-84e8-40b7-a037-594b97121e9b admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:47.483005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b755bc-49b2-420b-9bf3-10b34b39e283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:47.483758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b755bc-49b2-420b-9bf3-10b34b39e283 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:47.484113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b755bc-49b2-420b-9bf3-10b34b39e283 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:47.484881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b755bc-49b2-420b-9bf3-10b34b39e283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:47.496284 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b755bc-49b2-420b-9bf3-10b34b39e283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:47.497890 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b755bc-49b2-420b-9bf3-10b34b39e283 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:47.507768 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1089/2183] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:47.889931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2ad70-1172-4188-82c1-2447cdcfb577 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:47.891457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-afc2ad70-1172-4188-82c1-2447cdcfb577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:47.904230 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2ad70-1172-4188-82c1-2447cdcfb577 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:47.908891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:47.910528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:47.910528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:47.910528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] PATH_INFO: `/v3/projects/398968bb78824dc697ace2d523bc885e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:47.911467 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=398968bb78824dc697ace2d523bc885e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:47.913454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:48.006084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '398968bb78824dc697ace2d523bc885e', 'request_id': 'req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:48.012976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '398968bb78824dc697ace2d523bc885e', 'request_id': 'req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:48.019407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '398968bb78824dc697ace2d523bc885e', 'request_id': 'req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:48.088694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] The token cache is being invalidate because project 398968bb78824dc697ace2d523bc885e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:11:48.089007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:48.089634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4330f07f-abd1-4e1c-9e65-33bdf1ebaf87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:11:48.091431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1095/2184] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:11:47 2022] DELETE /identity/v3/projects/398968bb78824dc697ace2d523bc885e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:48.096172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106cebd-9ca5-495d-b783-af15714c06fe None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:48.096656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106cebd-9ca5-495d-b783-af15714c06fe None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:48.097005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106cebd-9ca5-495d-b783-af15714c06fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:48.118803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4106cebd-9ca5-495d-b783-af15714c06fe None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:48.120230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4106cebd-9ca5-495d-b783-af15714c06fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:48.139588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:48.183410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1090/2185] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:11:48.192082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d6fcf-055b-4cc4-a186-bd6880621148 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:48.192447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d6fcf-055b-4cc4-a186-bd6880621148 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:48.192821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d6fcf-055b-4cc4-a186-bd6880621148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:48.212280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-205d6fcf-055b-4cc4-a186-bd6880621148 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:48.213518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-205d6fcf-055b-4cc4-a186-bd6880621148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:48.221869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1096/2186] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:48.226043 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106cebd-9ca5-495d-b783-af15714c06fe None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:48.236459 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106cebd-9ca5-495d-b783-af15714c06fe None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:48.241037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9bc21-59c1-41e4-b9b9-32122c6065e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:48.241685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9bc21-59c1-41e4-b9b9-32122c6065e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:48.241969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9bc21-59c1-41e4-b9b9-32122c6065e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:48.242901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9bc21-59c1-41e4-b9b9-32122c6065e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:48.243628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9bc21-59c1-41e4-b9b9-32122c6065e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:11:48.244010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9bc21-59c1-41e4-b9b9-32122c6065e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:48.245976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9bc21-59c1-41e4-b9b9-32122c6065e0 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:48.257830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1091/2187] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:11:48.261966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d6fcf-055b-4cc4-a186-bd6880621148 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:48.271220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d6fcf-055b-4cc4-a186-bd6880621148 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:48.276745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171eb741-222b-45e6-9fc7-63169985d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:48.277552 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171eb741-222b-45e6-9fc7-63169985d176 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:48.277991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171eb741-222b-45e6-9fc7-63169985d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:48.278285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171eb741-222b-45e6-9fc7-63169985d176 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:48.278988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171eb741-222b-45e6-9fc7-63169985d176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:48.281185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171eb741-222b-45e6-9fc7-63169985d176 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:48.372229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1097/2188] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:11:48 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:11:48.376433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9bc21-59c1-41e4-b9b9-32122c6065e0 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:48.387501 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9bc21-59c1-41e4-b9b9-32122c6065e0 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:48.452531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58ddef-6772-440e-9165-a3b256888e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:48.453359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58ddef-6772-440e-9165-a3b256888e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:48.453646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58ddef-6772-440e-9165-a3b256888e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:48.453923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58ddef-6772-440e-9165-a3b256888e01 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:48.454626 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58ddef-6772-440e-9165-a3b256888e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:48.456544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58ddef-6772-440e-9165-a3b256888e01 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:48.558470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1092/2189] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:11:48 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:11:48.562520 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171eb741-222b-45e6-9fc7-63169985d176 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:48.572964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171eb741-222b-45e6-9fc7-63169985d176 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:48.580977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8abdfd-4090-4c13-832a-2e4f32dddc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:48.580977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8abdfd-4090-4c13-832a-2e4f32dddc58 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:48.580977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8abdfd-4090-4c13-832a-2e4f32dddc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:48.580977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8abdfd-4090-4c13-832a-2e4f32dddc58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:48.581329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8abdfd-4090-4c13-832a-2e4f32dddc58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:48.581329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8abdfd-4090-4c13-832a-2e4f32dddc58 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:48.593006 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1098/2190] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:11:48.597146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58ddef-6772-440e-9165-a3b256888e01 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:48.608104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58ddef-6772-440e-9165-a3b256888e01 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:48.613316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bff77e-a2fc-4481-91b3-ac3651adeb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:48.613977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bff77e-a2fc-4481-91b3-ac3651adeb9b admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:48.614258 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bff77e-a2fc-4481-91b3-ac3651adeb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:48.614533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bff77e-a2fc-4481-91b3-ac3651adeb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:48.615259 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bff77e-a2fc-4481-91b3-ac3651adeb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:48.616871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bff77e-a2fc-4481-91b3-ac3651adeb9b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:48.627865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1093/2191] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:11:48.641041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8abdfd-4090-4c13-832a-2e4f32dddc58 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:48.651466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8abdfd-4090-4c13-832a-2e4f32dddc58 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:48.656449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6a8f3-10e8-4c12-bd2d-0a71bb7c56cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:48.657280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a8f3-10e8-4c12-bd2d-0a71bb7c56cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:48.657563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a8f3-10e8-4c12-bd2d-0a71bb7c56cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:48.658110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a8f3-10e8-4c12-bd2d-0a71bb7c56cb admin admin] PATH_INFO: `/v3/projects/663dfd1b0df3405da05202a61760cf49/users/66ea02975030452bbce993e8b9ab6cca/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:48.685557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6a8f3-10e8-4c12-bd2d-0a71bb7c56cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=663dfd1b0df3405da05202a61760cf49, user_id=66ea02975030452bbce993e8b9ab6cca, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:48.688572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6a8f3-10e8-4c12-bd2d-0a71bb7c56cb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:48.771838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1099/2192] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:11:48 2022] PUT /identity/v3/projects/663dfd1b0df3405da05202a61760cf49/users/66ea02975030452bbce993e8b9ab6cca/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:11:48.782925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5f546-bd97-452e-8bed-631864328d69 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:48.783217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5f546-bd97-452e-8bed-631864328d69 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:48.783497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5f546-bd97-452e-8bed-631864328d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:48.811064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-10a5f546-bd97-452e-8bed-631864328d69 None None] MFA Rules not processed for user `66ea02975030452bbce993e8b9ab6cca`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:48.812294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-10a5f546-bd97-452e-8bed-631864328d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:48.881276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1094/2193] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:48.893619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6a8f3-10e8-4c12-bd2d-0a71bb7c56cb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:48.895261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-00f6a8f3-10e8-4c12-bd2d-0a71bb7c56cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:48.917873 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:48.971521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6a8f3-10e8-4c12-bd2d-0a71bb7c56cb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:48.975184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184f209-1858-4e9c-bae3-b3d1a11d6a51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:48.975825 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184f209-1858-4e9c-bae3-b3d1a11d6a51 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:48.976109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184f209-1858-4e9c-bae3-b3d1a11d6a51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:48.976383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184f209-1858-4e9c-bae3-b3d1a11d6a51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:48.986514 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184f209-1858-4e9c-bae3-b3d1a11d6a51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:48.987947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184f209-1858-4e9c-bae3-b3d1a11d6a51 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:48.997565 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1100/2194] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:52.057145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5f546-bd97-452e-8bed-631864328d69 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:52.058634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-10a5f546-bd97-452e-8bed-631864328d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:52.083870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:52.153664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5f546-bd97-452e-8bed-631864328d69 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:52.158562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3b6bd-247a-4058-bba0-db254f3ef4dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:52.159229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3b6bd-247a-4058-bba0-db254f3ef4dd service placement] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:52.159512 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3b6bd-247a-4058-bba0-db254f3ef4dd service placement] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:52.159804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3b6bd-247a-4058-bba0-db254f3ef4dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:52.162090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-38f3b6bd-247a-4058-bba0-db254f3ef4dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:52.174170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3b6bd-247a-4058-bba0-db254f3ef4dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:52.176250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3b6bd-247a-4058-bba0-db254f3ef4dd service placement] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:52.186706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1095/2195] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:54.481620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def917d-525c-4e5a-bc2b-f216c417250f None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:54.482050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def917d-525c-4e5a-bc2b-f216c417250f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:54.482262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def917d-525c-4e5a-bc2b-f216c417250f None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:54.484971 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1101/2196] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:11:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:11:54.494209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e07eb8-dc22-41c5-af9d-fc9144aca027 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:54.494682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e07eb8-dc22-41c5-af9d-fc9144aca027 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:54.494761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e07eb8-dc22-41c5-af9d-fc9144aca027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:54.549702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-67e07eb8-dc22-41c5-af9d-fc9144aca027 None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:54.549702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-67e07eb8-dc22-41c5-af9d-fc9144aca027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:54.625572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1096/2197] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:11:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:54.637148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def917d-525c-4e5a-bc2b-f216c417250f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:54.638821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4def917d-525c-4e5a-bc2b-f216c417250f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:54.699749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def917d-525c-4e5a-bc2b-f216c417250f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:54.703409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a0097-d52b-4d96-980f-2d0f69e9642b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b712bd7d8d4930a3bfc99581be8cb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:54.704047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0097-d52b-4d96-980f-2d0f69e9642b service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:54.704336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0097-d52b-4d96-980f-2d0f69e9642b service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:54.704613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0097-d52b-4d96-980f-2d0f69e9642b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:54.715158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a0097-d52b-4d96-980f-2d0f69e9642b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:54.716605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a0097-d52b-4d96-980f-2d0f69e9642b service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:54.728171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1102/2198] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:54.798135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a0929-297c-4083-bff3-5b77c4c6e193 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:54.798432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a0929-297c-4083-bff3-5b77c4c6e193 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:54.798728 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a0929-297c-4083-bff3-5b77c4c6e193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:54.821103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d14a0929-297c-4083-bff3-5b77c4c6e193 None None] MFA Rules not processed for user `6e0cc3e3c4804237b78241cca28ca689`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:54.822317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d14a0929-297c-4083-bff3-5b77c4c6e193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:54.873729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:54.887598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1097/2199] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:11:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:54.924523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbdcd3-998e-4a40-8594-6ce234713c54 None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:54.924921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbdcd3-998e-4a40-8594-6ce234713c54 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:54.925196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbdcd3-998e-4a40-8594-6ce234713c54 None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:54.926784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1103/2200] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:11:54.932522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404db00-dc6b-4e77-8c50-d7bfb99a1710 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:54.932949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404db00-dc6b-4e77-8c50-d7bfb99a1710 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:54.933294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404db00-dc6b-4e77-8c50-d7bfb99a1710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:54.956159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4404db00-dc6b-4e77-8c50-d7bfb99a1710 None None] MFA Rules not processed for user `887d2ad3cbf7459a8a59e7442ba9744e`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:54.957550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4404db00-dc6b-4e77-8c50-d7bfb99a1710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:55.017194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1098/2201] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:11:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:55.029266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbdcd3-998e-4a40-8594-6ce234713c54 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:55.031277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-34cbdcd3-998e-4a40-8594-6ce234713c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:55.047981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbdcd3-998e-4a40-8594-6ce234713c54 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:55.051893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4573bf-035c-4cb2-8093-1f01cc598f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:55.052304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4573bf-035c-4cb2-8093-1f01cc598f98 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:55.052596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4573bf-035c-4cb2-8093-1f01cc598f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:55.053028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4573bf-035c-4cb2-8093-1f01cc598f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:55.065829 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4573bf-035c-4cb2-8093-1f01cc598f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:55.067323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4573bf-035c-4cb2-8093-1f01cc598f98 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:55.081718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1104/2202] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:55.169749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3963-8c4e-4320-9623-e9270e316e37 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:55.170134 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3963-8c4e-4320-9623-e9270e316e37 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:55.170496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd3963-8c4e-4320-9623-e9270e316e37 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:55.172035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1099/2203] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:11:55.177681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e7a22-51f3-4e34-960b-69df76a4f181 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:55.177968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e7a22-51f3-4e34-960b-69df76a4f181 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:55.178252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e7a22-51f3-4e34-960b-69df76a4f181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:55.199688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d50e7a22-51f3-4e34-960b-69df76a4f181 None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:55.200954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d50e7a22-51f3-4e34-960b-69df76a4f181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:55.212993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1105/2204] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:11:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:55.230597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3963-8c4e-4320-9623-e9270e316e37 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:55.245537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd3963-8c4e-4320-9623-e9270e316e37 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:55.250472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34eaee-049f-4d07-bbe3-ba7f5fa37011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:55.251122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34eaee-049f-4d07-bbe3-ba7f5fa37011 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:55.251404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34eaee-049f-4d07-bbe3-ba7f5fa37011 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:55.251677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34eaee-049f-4d07-bbe3-ba7f5fa37011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:55.264282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34eaee-049f-4d07-bbe3-ba7f5fa37011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:55.265775 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34eaee-049f-4d07-bbe3-ba7f5fa37011 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:55.279740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1100/2205] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:11:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:11:59.582520 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b82696-467d-41a6-812a-111a596037e1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:59.582996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b82696-467d-41a6-812a-111a596037e1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:59.583100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b82696-467d-41a6-812a-111a596037e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:59.634238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-86b82696-467d-41a6-812a-111a596037e1 None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:11:59.635471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-86b82696-467d-41a6-812a-111a596037e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:59.696725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:11:59.701817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1106/2206] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:11:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:11:59.713900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34eaee-049f-4d07-bbe3-ba7f5fa37011 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:11:59.715279 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7f34eaee-049f-4d07-bbe3-ba7f5fa37011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:11:59.808457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34eaee-049f-4d07-bbe3-ba7f5fa37011 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:11:59.814218 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778df6b-ffc0-4ac4-a332-9de3172373aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:11:59.814862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778df6b-ffc0-4ac4-a332-9de3172373aa service swift] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:11:59.815167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778df6b-ffc0-4ac4-a332-9de3172373aa service swift] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:11:59.815453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778df6b-ffc0-4ac4-a332-9de3172373aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:11:59.824968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778df6b-ffc0-4ac4-a332-9de3172373aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:11:59.826342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778df6b-ffc0-4ac4-a332-9de3172373aa service swift] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:11:59.834711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1101/2207] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:11:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:12:00.477499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b82696-467d-41a6-812a-111a596037e1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:00.608945 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b82696-467d-41a6-812a-111a596037e1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:00.614696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:00.615721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:00.616077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:00.616397 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] PATH_INFO: `/v3/users/66ea02975030452bbce993e8b9ab6cca` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:00.618425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ea02975030452bbce993e8b9ab6cca)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:00.620939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:00.817534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ea02975030452bbce993e8b9ab6cca'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:00.825584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ea02975030452bbce993e8b9ab6cca'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:00.826370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ea02975030452bbce993e8b9ab6cca'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:00.827034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ea02975030452bbce993e8b9ab6cca'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:00.841917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ea02975030452bbce993e8b9ab6cca'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:00.920487 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1107/2208] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:12:00 2022] DELETE /identity/v3/users/66ea02975030452bbce993e8b9ab6cca => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:12:00.925601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d05b4-d4b4-45c5-bfec-6f1525c54e77 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:00.926084 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d05b4-d4b4-45c5-bfec-6f1525c54e77 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:00.926203 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d05b4-d4b4-45c5-bfec-6f1525c54e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:00.944325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-809d05b4-d4b4-45c5-bfec-6f1525c54e77 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:00.945794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-809d05b4-d4b4-45c5-bfec-6f1525c54e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:01.001739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:01.012066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1102/2209] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:01.021738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:01.023250 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:01.092864 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d9dd0-f62f-485e-bd9a-ef22e8af4469 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:01.096581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19594a1-4154-4ead-8e0e-0d399b8f28ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:01.097275 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19594a1-4154-4ead-8e0e-0d399b8f28ee service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:01.097556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19594a1-4154-4ead-8e0e-0d399b8f28ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:01.097828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19594a1-4154-4ead-8e0e-0d399b8f28ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:01.107617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19594a1-4154-4ead-8e0e-0d399b8f28ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:01.109111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19594a1-4154-4ead-8e0e-0d399b8f28ee service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:01.119635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1108/2210] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:12:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:12:01.562265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d05b4-d4b4-45c5-bfec-6f1525c54e77 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:01.563764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-809d05b4-d4b4-45c5-bfec-6f1525c54e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:01.576983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d05b4-d4b4-45c5-bfec-6f1525c54e77 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:01.585817 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:01.589187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:01.589974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:01.591060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] PATH_INFO: `/v3/projects/663dfd1b0df3405da05202a61760cf49` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:01.593342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=663dfd1b0df3405da05202a61760cf49)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:01.597296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:01.734649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '663dfd1b0df3405da05202a61760cf49', 'request_id': 'req-9ba2b625-2875-463e-8399-d867fbd138ff'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:01.741592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '663dfd1b0df3405da05202a61760cf49', 'request_id': 'req-9ba2b625-2875-463e-8399-d867fbd138ff'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:01.827751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '663dfd1b0df3405da05202a61760cf49', 'request_id': 'req-9ba2b625-2875-463e-8399-d867fbd138ff'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:01.845220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] The token cache is being invalidate because project 663dfd1b0df3405da05202a61760cf49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:12:01.845687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:01.846490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-9ba2b625-2875-463e-8399-d867fbd138ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:01.848568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1103/2211] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:12:01 2022] DELETE /identity/v3/projects/663dfd1b0df3405da05202a61760cf49 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:12:01.854319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d52ddb-194d-4acd-a3c2-c07fb0b7ebfa None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:01.854867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d52ddb-194d-4acd-a3c2-c07fb0b7ebfa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:01.855345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d52ddb-194d-4acd-a3c2-c07fb0b7ebfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:01.877287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-94d52ddb-194d-4acd-a3c2-c07fb0b7ebfa None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:01.878670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-94d52ddb-194d-4acd-a3c2-c07fb0b7ebfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:01.905502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:01.957704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1109/2212] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:01.965782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b08dc0-c0c6-4b77-a6c5-689d55cce0f2 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:01.966416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b08dc0-c0c6-4b77-a6c5-689d55cce0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:01.966416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b08dc0-c0c6-4b77-a6c5-689d55cce0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:01.986919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c3b08dc0-c0c6-4b77-a6c5-689d55cce0f2 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:01.988220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c3b08dc0-c0c6-4b77-a6c5-689d55cce0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:02.000324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1104/2213] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:12:02.002315 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d52ddb-194d-4acd-a3c2-c07fb0b7ebfa None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:02.012679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d52ddb-194d-4acd-a3c2-c07fb0b7ebfa None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:02.018156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0374219-5a92-421b-8658-34c9016b5be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:02.019038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0374219-5a92-421b-8658-34c9016b5be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:02.019324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0374219-5a92-421b-8658-34c9016b5be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:02.019734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0374219-5a92-421b-8658-34c9016b5be6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:02.020704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0374219-5a92-421b-8658-34c9016b5be6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:12:02.021044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0374219-5a92-421b-8658-34c9016b5be6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:02.022796 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0374219-5a92-421b-8658-34c9016b5be6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:02.036902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1110/2214] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:12:02.041737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b08dc0-c0c6-4b77-a6c5-689d55cce0f2 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:02.055163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b08dc0-c0c6-4b77-a6c5-689d55cce0f2 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:02.060067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4787c-6abc-4d2f-b682-980f07f008cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:02.060723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4787c-6abc-4d2f-b682-980f07f008cc admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:02.061097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4787c-6abc-4d2f-b682-980f07f008cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:02.061371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4787c-6abc-4d2f-b682-980f07f008cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:02.062060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4787c-6abc-4d2f-b682-980f07f008cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:02.063948 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4787c-6abc-4d2f-b682-980f07f008cc admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:02.168837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1105/2215] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:12:02 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:12:02.172207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0374219-5a92-421b-8658-34c9016b5be6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:02.181150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0374219-5a92-421b-8658-34c9016b5be6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:02.242231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02158be7-477a-4ffd-9dc1-a1357dc226ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:02.242901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158be7-477a-4ffd-9dc1-a1357dc226ba admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:02.243189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158be7-477a-4ffd-9dc1-a1357dc226ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:02.243468 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158be7-477a-4ffd-9dc1-a1357dc226ba admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:02.244175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02158be7-477a-4ffd-9dc1-a1357dc226ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:02.246133 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02158be7-477a-4ffd-9dc1-a1357dc226ba admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:02.334663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1111/2216] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:12:02 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:12:02.339472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4787c-6abc-4d2f-b682-980f07f008cc admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:02.352361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4787c-6abc-4d2f-b682-980f07f008cc admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:02.357233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b65fa-01b2-4c8c-8f33-49d4404d07f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:02.357902 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b65fa-01b2-4c8c-8f33-49d4404d07f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:02.358208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b65fa-01b2-4c8c-8f33-49d4404d07f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:02.359155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b65fa-01b2-4c8c-8f33-49d4404d07f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:02.359729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b65fa-01b2-4c8c-8f33-49d4404d07f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:02.361410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b65fa-01b2-4c8c-8f33-49d4404d07f5 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:02.372455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1106/2217] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:12:02.376788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02158be7-477a-4ffd-9dc1-a1357dc226ba admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:02.387724 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02158be7-477a-4ffd-9dc1-a1357dc226ba admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:02.392789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7119487-1665-4e55-a86b-9d93f0dfc086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:02.393617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7119487-1665-4e55-a86b-9d93f0dfc086 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:02.394001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7119487-1665-4e55-a86b-9d93f0dfc086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:02.394376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7119487-1665-4e55-a86b-9d93f0dfc086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:02.395323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7119487-1665-4e55-a86b-9d93f0dfc086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:02.397013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7119487-1665-4e55-a86b-9d93f0dfc086 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:02.409124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1112/2218] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:12:02.415112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b65fa-01b2-4c8c-8f33-49d4404d07f5 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:02.426282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b65fa-01b2-4c8c-8f33-49d4404d07f5 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:02.431976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46071d8b-7a83-47f6-8893-4f284a324646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:02.432921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46071d8b-7a83-47f6-8893-4f284a324646 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:02.433208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46071d8b-7a83-47f6-8893-4f284a324646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:02.433487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46071d8b-7a83-47f6-8893-4f284a324646 admin admin] PATH_INFO: `/v3/projects/71b68c04578747d08842aab556e98030/users/80de40d2a02343858849d481590aa570/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:02.448651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46071d8b-7a83-47f6-8893-4f284a324646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b68c04578747d08842aab556e98030, user_id=80de40d2a02343858849d481590aa570, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:02.451364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46071d8b-7a83-47f6-8893-4f284a324646 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:02.518675 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1107/2219] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:12:02 2022] PUT /identity/v3/projects/71b68c04578747d08842aab556e98030/users/80de40d2a02343858849d481590aa570/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:12:02.526702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f05877-3ef2-490e-ab87-b1797dfb7aec None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:02.527233 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f05877-3ef2-490e-ab87-b1797dfb7aec None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:02.527590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f05877-3ef2-490e-ab87-b1797dfb7aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:02.546658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-42f05877-3ef2-490e-ab87-b1797dfb7aec None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:02.548026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-42f05877-3ef2-490e-ab87-b1797dfb7aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:02.622015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1113/2220] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:02.629767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46071d8b-7a83-47f6-8893-4f284a324646 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:02.632532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-46071d8b-7a83-47f6-8893-4f284a324646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:02.694372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46071d8b-7a83-47f6-8893-4f284a324646 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:02.698458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32477904-c4bb-460d-9599-cd495e02ded1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:02.699101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32477904-c4bb-460d-9599-cd495e02ded1 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:02.699383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32477904-c4bb-460d-9599-cd495e02ded1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:02.699659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32477904-c4bb-460d-9599-cd495e02ded1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:02.710040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32477904-c4bb-460d-9599-cd495e02ded1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:02.712205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32477904-c4bb-460d-9599-cd495e02ded1 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:02.722685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1108/2221] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:12:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:12:10.166254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33587e-b7b0-4f54-ab0e-4b86306204d1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:10.166834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33587e-b7b0-4f54-ab0e-4b86306204d1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:10.167066 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33587e-b7b0-4f54-ab0e-4b86306204d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:10.198051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1c33587e-b7b0-4f54-ab0e-4b86306204d1 None None] MFA Rules not processed for user `80de40d2a02343858849d481590aa570`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:10.199616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1c33587e-b7b0-4f54-ab0e-4b86306204d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:10.215988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:10.270347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1114/2222] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:10.275283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32477904-c4bb-460d-9599-cd495e02ded1 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:10.287892 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32477904-c4bb-460d-9599-cd495e02ded1 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:10.293116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b3803-c508-4595-abb8-c7be9f1179eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:10.293774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b3803-c508-4595-abb8-c7be9f1179eb admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:10.294055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b3803-c508-4595-abb8-c7be9f1179eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:10.294329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b3803-c508-4595-abb8-c7be9f1179eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:10.295340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b3803-c508-4595-abb8-c7be9f1179eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:10.297443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b3803-c508-4595-abb8-c7be9f1179eb admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:10.405475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:10.412918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1109/2223] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:12:10 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:12:10.417155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33587e-b7b0-4f54-ab0e-4b86306204d1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:10.427754 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33587e-b7b0-4f54-ab0e-4b86306204d1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:10.491900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fa096-2f55-4cf1-9b21-e8e9b819bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:10.492570 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fa096-2f55-4cf1-9b21-e8e9b819bd39 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:10.492881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fa096-2f55-4cf1-9b21-e8e9b819bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:10.493161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fa096-2f55-4cf1-9b21-e8e9b819bd39 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:10.493874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fa096-2f55-4cf1-9b21-e8e9b819bd39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:10.495951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fa096-2f55-4cf1-9b21-e8e9b819bd39 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:10.629431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1115/2224] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:12:10 2022] POST /identity/v3/users => generated 298 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:12:10.634597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b3803-c508-4595-abb8-c7be9f1179eb admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:10.646487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b3803-c508-4595-abb8-c7be9f1179eb admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:10.651761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea43c62-3b36-4159-b22f-3ffcaa01ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:10.652431 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea43c62-3b36-4159-b22f-3ffcaa01ba61 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:10.652716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea43c62-3b36-4159-b22f-3ffcaa01ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:10.653041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea43c62-3b36-4159-b22f-3ffcaa01ba61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:10.653772 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea43c62-3b36-4159-b22f-3ffcaa01ba61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:10.655368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea43c62-3b36-4159-b22f-3ffcaa01ba61 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:10.671440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1110/2225] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:12:10.672718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fa096-2f55-4cf1-9b21-e8e9b819bd39 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:10.686461 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fa096-2f55-4cf1-9b21-e8e9b819bd39 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:10.691392 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5191c-444e-42d4-a1c7-c9863ff8e6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:10.692328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5191c-444e-42d4-a1c7-c9863ff8e6a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:10.692556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5191c-444e-42d4-a1c7-c9863ff8e6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:10.692865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5191c-444e-42d4-a1c7-c9863ff8e6a2 admin admin] PATH_INFO: `/v3/projects/c763e54bb34246659fca207127eb7a9f/users/13de76cf9f5742e0bda36a6dee992410/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:10.712528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5191c-444e-42d4-a1c7-c9863ff8e6a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c763e54bb34246659fca207127eb7a9f, user_id=13de76cf9f5742e0bda36a6dee992410, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:10.715454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5191c-444e-42d4-a1c7-c9863ff8e6a2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:10.832905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1116/2226] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:12:10 2022] PUT /identity/v3/projects/c763e54bb34246659fca207127eb7a9f/users/13de76cf9f5742e0bda36a6dee992410/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:12:10.838891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea43c62-3b36-4159-b22f-3ffcaa01ba61 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:10.848204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea43c62-3b36-4159-b22f-3ffcaa01ba61 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:10.911638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cbcf0-fde8-459e-80f4-252ebefa2ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:10.912309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cbcf0-fde8-459e-80f4-252ebefa2ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:10.912594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cbcf0-fde8-459e-80f4-252ebefa2ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:10.912913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cbcf0-fde8-459e-80f4-252ebefa2ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:10.913641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cbcf0-fde8-459e-80f4-252ebefa2ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:10.915263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cbcf0-fde8-459e-80f4-252ebefa2ceb admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:10.925766 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1111/2227] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:12:10.930336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5191c-444e-42d4-a1c7-c9863ff8e6a2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:10.941693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5191c-444e-42d4-a1c7-c9863ff8e6a2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:10.948199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9043-d0c5-4710-b7e5-5e4198763367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:10.949162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9043-d0c5-4710-b7e5-5e4198763367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:10.949448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9043-d0c5-4710-b7e5-5e4198763367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:10.949725 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9043-d0c5-4710-b7e5-5e4198763367 admin admin] PATH_INFO: `/v3/projects/c763e54bb34246659fca207127eb7a9f/users/13de76cf9f5742e0bda36a6dee992410/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:10.951734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea9043-d0c5-4710-b7e5-5e4198763367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c763e54bb34246659fca207127eb7a9f, user_id=13de76cf9f5742e0bda36a6dee992410, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:10.954439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea9043-d0c5-4710-b7e5-5e4198763367 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:10.966504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1117/2228] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:12:10 2022] PUT /identity/v3/projects/c763e54bb34246659fca207127eb7a9f/users/13de76cf9f5742e0bda36a6dee992410/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:12:16.831008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d7029-0941-4504-aa63-6a1303d6e97a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:16.831553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d7029-0941-4504-aa63-6a1303d6e97a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:16.831553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d7029-0941-4504-aa63-6a1303d6e97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:16.863012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e06d7029-0941-4504-aa63-6a1303d6e97a None None] MFA Rules not processed for user `13de76cf9f5742e0bda36a6dee992410`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:16.864144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e06d7029-0941-4504-aa63-6a1303d6e97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:16.938367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1112/2229] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:16.961472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9043-d0c5-4710-b7e5-5e4198763367 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:16.963364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-50ea9043-d0c5-4710-b7e5-5e4198763367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:17.048664 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:17.051578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9043-d0c5-4710-b7e5-5e4198763367 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:17.055316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfc85b-1a22-4284-8fda-1a88302edd6d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b712bd7d8d4930a3bfc99581be8cb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:17.055968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfc85b-1a22-4284-8fda-1a88302edd6d service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:17.056254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfc85b-1a22-4284-8fda-1a88302edd6d service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:17.056535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfc85b-1a22-4284-8fda-1a88302edd6d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:17.119874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbfc85b-1a22-4284-8fda-1a88302edd6d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:17.121317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbfc85b-1a22-4284-8fda-1a88302edd6d service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:17.130771 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1118/2230] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:12:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:12:17.370852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825c773-fa01-4ae6-b5ff-82299e01d4cb None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:17.371076 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825c773-fa01-4ae6-b5ff-82299e01d4cb None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:17.371360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825c773-fa01-4ae6-b5ff-82299e01d4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:17.417640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f825c773-fa01-4ae6-b5ff-82299e01d4cb None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:17.418881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f825c773-fa01-4ae6-b5ff-82299e01d4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:17.497567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1113/2231] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:12:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:17.510304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfc85b-1a22-4284-8fda-1a88302edd6d service glance] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:17.511977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5fbfc85b-1a22-4284-8fda-1a88302edd6d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:17.609672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfc85b-1a22-4284-8fda-1a88302edd6d service glance] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:17.616158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f329cd-545c-4a98-9cba-d751058004e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:17.617511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f329cd-545c-4a98-9cba-d751058004e9 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:17.617810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f329cd-545c-4a98-9cba-d751058004e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:17.618331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f329cd-545c-4a98-9cba-d751058004e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:17.641102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f329cd-545c-4a98-9cba-d751058004e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:17.641102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f329cd-545c-4a98-9cba-d751058004e9 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:17.650440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1119/2232] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:12:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:12:25.028959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58a78a-c3b0-49bb-85e0-d766c81d655a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:25.029546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58a78a-c3b0-49bb-85e0-d766c81d655a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:25.029546 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58a78a-c3b0-49bb-85e0-d766c81d655a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:25.069991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-bd58a78a-c3b0-49bb-85e0-d766c81d655a None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:25.071151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-bd58a78a-c3b0-49bb-85e0-d766c81d655a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:25.087060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1114/2233] 10.222.0.45 () {58 vars in 967 bytes} [Tue Jul 5 16:12:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:25.106416 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f329cd-545c-4a98-9cba-d751058004e9 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:25.121004 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f329cd-545c-4a98-9cba-d751058004e9 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:25.129115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d65f6f-5ea8-4d5d-905d-25a80e9ade19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:25.129815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d65f6f-5ea8-4d5d-905d-25a80e9ade19 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:25.130111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d65f6f-5ea8-4d5d-905d-25a80e9ade19 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:25.130396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d65f6f-5ea8-4d5d-905d-25a80e9ade19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:25.141177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d65f6f-5ea8-4d5d-905d-25a80e9ade19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:25.143792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d65f6f-5ea8-4d5d-905d-25a80e9ade19 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:25.155837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1120/2234] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:12:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:12:31.091396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaa648-f93e-4493-aef9-c8219f57a338 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:31.091956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaa648-f93e-4493-aef9-c8219f57a338 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:31.091956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaa648-f93e-4493-aef9-c8219f57a338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:31.103762 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:31.109332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-adeaa648-f93e-4493-aef9-c8219f57a338 None None] MFA Rules not processed for user `80de40d2a02343858849d481590aa570`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:31.110764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-adeaa648-f93e-4493-aef9-c8219f57a338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:31.117502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1115/2235] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:33.159816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b8e7e-380c-400a-8b3c-a0da33231e3e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:33.160355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b8e7e-380c-400a-8b3c-a0da33231e3e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:33.160464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b8e7e-380c-400a-8b3c-a0da33231e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:33.179608 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-fe8b8e7e-380c-400a-8b3c-a0da33231e3e None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:33.180830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fe8b8e7e-380c-400a-8b3c-a0da33231e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:33.188175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1121/2236] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:12:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:33.209847 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaa648-f93e-4493-aef9-c8219f57a338 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:33.221808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaa648-f93e-4493-aef9-c8219f57a338 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:33.227209 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04f3ac-41f3-405f-91f2-42ae58f66d8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:33.227843 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04f3ac-41f3-405f-91f2-42ae58f66d8c service swift] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:33.228129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04f3ac-41f3-405f-91f2-42ae58f66d8c service swift] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:33.228408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04f3ac-41f3-405f-91f2-42ae58f66d8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:33.236724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04f3ac-41f3-405f-91f2-42ae58f66d8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:33.238353 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04f3ac-41f3-405f-91f2-42ae58f66d8c service swift] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:33.246970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1116/2237] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:12:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:12:35.620491 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0962b7-d025-4f86-a334-b95c24a34ba8 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:35.621555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0962b7-d025-4f86-a334-b95c24a34ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:35.622115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0962b7-d025-4f86-a334-b95c24a34ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:35.665635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5d0962b7-d025-4f86-a334-b95c24a34ba8 None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:35.667280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5d0962b7-d025-4f86-a334-b95c24a34ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:35.690126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1122/2238] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:12:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:35.782173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04f3ac-41f3-405f-91f2-42ae58f66d8c service swift] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:35.799184 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04f3ac-41f3-405f-91f2-42ae58f66d8c service swift] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:35.815676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535faff-b2d4-4580-b0ef-c59b943e2b04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:35.816672 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535faff-b2d4-4580-b0ef-c59b943e2b04 service swift] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:35.817133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535faff-b2d4-4580-b0ef-c59b943e2b04 service swift] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:35.817617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535faff-b2d4-4580-b0ef-c59b943e2b04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:35.827411 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535faff-b2d4-4580-b0ef-c59b943e2b04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:35.829314 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535faff-b2d4-4580-b0ef-c59b943e2b04 service swift] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:35.838476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1117/2239] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:12:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:12:45.651735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=70 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:45.837431 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=12 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:49.999912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490fea8-cb28-46eb-b033-c96eb9b5fe58 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:50.000808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490fea8-cb28-46eb-b033-c96eb9b5fe58 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:50.000808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490fea8-cb28-46eb-b033-c96eb9b5fe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:50.038061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b490fea8-cb28-46eb-b033-c96eb9b5fe58 None None] MFA Rules not processed for user `0c8512e7c1704fac9cc4fcfe3bf1f18b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:50.039471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b490fea8-cb28-46eb-b033-c96eb9b5fe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:50.168411 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1123/2240] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:53.643157 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da460c75-6e28-46b2-aa10-c56b9e0862a0 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:53.643701 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da460c75-6e28-46b2-aa10-c56b9e0862a0 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:53.643701 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da460c75-6e28-46b2-aa10-c56b9e0862a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:53.713680 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-da460c75-6e28-46b2-aa10-c56b9e0862a0 None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:53.714790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-da460c75-6e28-46b2-aa10-c56b9e0862a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:53.722039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1118/2241] 10.222.0.45 () {58 vars in 967 bytes} [Tue Jul 5 16:12:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:53.735169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490fea8-cb28-46eb-b033-c96eb9b5fe58 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:53.747860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490fea8-cb28-46eb-b033-c96eb9b5fe58 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:53.752832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b30b5e-c8bd-4916-826c-816dc423d047 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:53.753498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b30b5e-c8bd-4916-826c-816dc423d047 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:53.753784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b30b5e-c8bd-4916-826c-816dc423d047 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:53.756112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b30b5e-c8bd-4916-826c-816dc423d047 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:53.765529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b30b5e-c8bd-4916-826c-816dc423d047 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:53.767143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b30b5e-c8bd-4916-826c-816dc423d047 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:53.775740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1124/2242] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:12:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:12:56.955163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da460c75-6e28-46b2-aa10-c56b9e0862a0 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:56.957763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-da460c75-6e28-46b2-aa10-c56b9e0862a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:57.035871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da460c75-6e28-46b2-aa10-c56b9e0862a0 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:57.040899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:57.042024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:57.042402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:57.042907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] PATH_INFO: `/v3/users/6e0cc3e3c4804237b78241cca28ca689` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:57.044513 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0cc3e3c4804237b78241cca28ca689)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:57.046980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:57.242300 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0cc3e3c4804237b78241cca28ca689'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:57.249949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0cc3e3c4804237b78241cca28ca689'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:57.250732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0cc3e3c4804237b78241cca28ca689'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:57.272299 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0cc3e3c4804237b78241cca28ca689'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:57.273296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-65ca5cf9-9429-4d49-b66f-2ec0c6c76e95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0cc3e3c4804237b78241cca28ca689'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:57.358212 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1119/2243] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:12:56 2022] DELETE /identity/v3/users/6e0cc3e3c4804237b78241cca28ca689 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:12:57.629858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b30b5e-c8bd-4916-826c-816dc423d047 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:57.644525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b30b5e-c8bd-4916-826c-816dc423d047 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:57.731487 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:57.732371 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:57.732662 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:57.732986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] PATH_INFO: `/v3/projects/dce6460ca0714dc98df6e69034242a7f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:57.734209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce6460ca0714dc98df6e69034242a7f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:57.736822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:57.926283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce6460ca0714dc98df6e69034242a7f', 'request_id': 'req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:57.932630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce6460ca0714dc98df6e69034242a7f', 'request_id': 'req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:57.939407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce6460ca0714dc98df6e69034242a7f', 'request_id': 'req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:58.052744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] The token cache is being invalidate because project dce6460ca0714dc98df6e69034242a7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:12:58.053189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:58.053807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-96e2d601-066f-4f73-a07b-0f7b26aa2cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:58.055333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1125/2244] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:12:57 2022] DELETE /identity/v3/projects/dce6460ca0714dc98df6e69034242a7f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:12:58.060105 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e437130-8619-491f-bbd8-48e04f67927d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:58.060571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e437130-8619-491f-bbd8-48e04f67927d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:58.060677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e437130-8619-491f-bbd8-48e04f67927d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:58.078709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-5e437130-8619-491f-bbd8-48e04f67927d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:58.079921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5e437130-8619-491f-bbd8-48e04f67927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:58.139783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1120/2245] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:58.203389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea80ac3-31e6-4062-b4c5-fdb1257c8665 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:58.203681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea80ac3-31e6-4062-b4c5-fdb1257c8665 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:58.203963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea80ac3-31e6-4062-b4c5-fdb1257c8665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:58.222025 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2ea80ac3-31e6-4062-b4c5-fdb1257c8665 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:58.223171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2ea80ac3-31e6-4062-b4c5-fdb1257c8665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:58.231573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1126/2246] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:58.235993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e437130-8619-491f-bbd8-48e04f67927d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:58.246367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e437130-8619-491f-bbd8-48e04f67927d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:58.250928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f46882-5c3f-4d1f-848b-1be50d2fb4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:58.251579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f46882-5c3f-4d1f-848b-1be50d2fb4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:58.251872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f46882-5c3f-4d1f-848b-1be50d2fb4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:58.252153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f46882-5c3f-4d1f-848b-1be50d2fb4c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:58.252924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f46882-5c3f-4d1f-848b-1be50d2fb4c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:12:58.253239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f46882-5c3f-4d1f-848b-1be50d2fb4c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:58.254826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f46882-5c3f-4d1f-848b-1be50d2fb4c7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:58.263332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:58.266561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1121/2247] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:12:58.270677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea80ac3-31e6-4062-b4c5-fdb1257c8665 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:58.279962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea80ac3-31e6-4062-b4c5-fdb1257c8665 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:58.284624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aef998-0764-4feb-ad6a-10c2b6e3b482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:58.285309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aef998-0764-4feb-ad6a-10c2b6e3b482 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:58.285595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aef998-0764-4feb-ad6a-10c2b6e3b482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:58.285874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aef998-0764-4feb-ad6a-10c2b6e3b482 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:58.287110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aef998-0764-4feb-ad6a-10c2b6e3b482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:58.289115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aef998-0764-4feb-ad6a-10c2b6e3b482 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:58.303121 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:58.385205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1127/2248] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:12:58 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:12:58.389855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f46882-5c3f-4d1f-848b-1be50d2fb4c7 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:58.400412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f46882-5c3f-4d1f-848b-1be50d2fb4c7 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:58.456460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d181d-19d5-4f81-9799-296d81c53358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:58.457155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d181d-19d5-4f81-9799-296d81c53358 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:58.457444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d181d-19d5-4f81-9799-296d81c53358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:58.457724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d181d-19d5-4f81-9799-296d81c53358 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:58.461123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d181d-19d5-4f81-9799-296d81c53358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:58.461123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d181d-19d5-4f81-9799-296d81c53358 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:58.535250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1122/2249] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:12:58 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:12:58.540055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aef998-0764-4feb-ad6a-10c2b6e3b482 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:58.550265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aef998-0764-4feb-ad6a-10c2b6e3b482 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:58.555238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808126f-86c4-4542-843a-6daa4a90fd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:58.556176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808126f-86c4-4542-843a-6daa4a90fd99 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:58.556463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808126f-86c4-4542-843a-6daa4a90fd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:58.556741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808126f-86c4-4542-843a-6daa4a90fd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:58.557787 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808126f-86c4-4542-843a-6daa4a90fd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:58.559387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808126f-86c4-4542-843a-6daa4a90fd99 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:58.570880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1128/2250] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:12:58.574807 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d181d-19d5-4f81-9799-296d81c53358 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:58.586233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d181d-19d5-4f81-9799-296d81c53358 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:58.590973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105f375-76b8-49b6-9844-8fa1fb8d217b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:58.591635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105f375-76b8-49b6-9844-8fa1fb8d217b admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:58.592058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105f375-76b8-49b6-9844-8fa1fb8d217b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:58.592357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105f375-76b8-49b6-9844-8fa1fb8d217b admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:58.593123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105f375-76b8-49b6-9844-8fa1fb8d217b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:58.594794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105f375-76b8-49b6-9844-8fa1fb8d217b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:58.606235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1123/2251] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:12:58.610659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808126f-86c4-4542-843a-6daa4a90fd99 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:58.620739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808126f-86c4-4542-843a-6daa4a90fd99 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:58.625623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02ae6c-65fd-4df6-8cce-41d5ec4882da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:58.626504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02ae6c-65fd-4df6-8cce-41d5ec4882da admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:58.626792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02ae6c-65fd-4df6-8cce-41d5ec4882da admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:58.627076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02ae6c-65fd-4df6-8cce-41d5ec4882da admin admin] PATH_INFO: `/v3/projects/3989c95816d542ae9f002e2b5e5338a7/users/0e491c68704445d18990526dec3a860b/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:58.643940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02ae6c-65fd-4df6-8cce-41d5ec4882da admin admin] RBAC: Authorizing `identity:create_grant(project_id=3989c95816d542ae9f002e2b5e5338a7, user_id=0e491c68704445d18990526dec3a860b, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:58.647208 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02ae6c-65fd-4df6-8cce-41d5ec4882da admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:58.722375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1129/2252] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:12:58 2022] PUT /identity/v3/projects/3989c95816d542ae9f002e2b5e5338a7/users/0e491c68704445d18990526dec3a860b/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:12:58.733081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efacd61-f577-457f-adf0-43a2368776ff None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:58.733375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efacd61-f577-457f-adf0-43a2368776ff None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:58.733656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efacd61-f577-457f-adf0-43a2368776ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:58.765111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8efacd61-f577-457f-adf0-43a2368776ff None None] MFA Rules not processed for user `0e491c68704445d18990526dec3a860b`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:58.766338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8efacd61-f577-457f-adf0-43a2368776ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:58.840913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1124/2253] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:58.852141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02ae6c-65fd-4df6-8cce-41d5ec4882da admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:58.853937 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eb02ae6c-65fd-4df6-8cce-41d5ec4882da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:58.910462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02ae6c-65fd-4df6-8cce-41d5ec4882da admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:58.914114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965390f-22e1-4ce6-93ec-7e80402e8edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:58.914755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965390f-22e1-4ce6-93ec-7e80402e8edf service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:58.915045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965390f-22e1-4ce6-93ec-7e80402e8edf service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:58.915328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965390f-22e1-4ce6-93ec-7e80402e8edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:58.923521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965390f-22e1-4ce6-93ec-7e80402e8edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:58.925056 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965390f-22e1-4ce6-93ec-7e80402e8edf service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:58.933815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1130/2254] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:12:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:12:58.959010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efacd61-f577-457f-adf0-43a2368776ff None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:59.020216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efacd61-f577-457f-adf0-43a2368776ff None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:59.025286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:59.026145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:59.026427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:59.026703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] PATH_INFO: `/v3/users/0e491c68704445d18990526dec3a860b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:59.028064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e491c68704445d18990526dec3a860b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:59.029875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:59.041529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:12:59.214609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e491c68704445d18990526dec3a860b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:59.222248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e491c68704445d18990526dec3a860b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:59.222930 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e491c68704445d18990526dec3a860b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:59.235469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e491c68704445d18990526dec3a860b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:59.236113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e491c68704445d18990526dec3a860b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:12:59.321408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1125/2255] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:12:58 2022] DELETE /identity/v3/users/0e491c68704445d18990526dec3a860b => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:12:59.326030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720422a-ca7a-4d42-b862-937e38c7b064 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:59.326321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720422a-ca7a-4d42-b862-937e38c7b064 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:59.326602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720422a-ca7a-4d42-b862-937e38c7b064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:59.344198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-2720422a-ca7a-4d42-b862-937e38c7b064 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:12:59.345712 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2720422a-ca7a-4d42-b862-937e38c7b064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:59.403031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1131/2256] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:12:59.412555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:59.413971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:59.471691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cd778-386e-47f7-9cf5-4c6c25050c47 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:59.475827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8050c38-4296-4b0e-957e-00f774275fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:59.476458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8050c38-4296-4b0e-957e-00f774275fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:59.476737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8050c38-4296-4b0e-957e-00f774275fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:59.477038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8050c38-4296-4b0e-957e-00f774275fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:59.486102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8050c38-4296-4b0e-957e-00f774275fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:59.487515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8050c38-4296-4b0e-957e-00f774275fd5 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:12:59.497369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1126/2257] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:12:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:12:59.970620 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720422a-ca7a-4d42-b862-937e38c7b064 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:12:59.972198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2720422a-ca7a-4d42-b862-937e38c7b064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:12:59.984086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720422a-ca7a-4d42-b862-937e38c7b064 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:12:59.989089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:12:59.990474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:12:59.990751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:12:59.991026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] PATH_INFO: `/v3/projects/3989c95816d542ae9f002e2b5e5338a7` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:12:59.992224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3989c95816d542ae9f002e2b5e5338a7)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:12:59.994191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:13:00.094374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3989c95816d542ae9f002e2b5e5338a7', 'request_id': 'req-45f78a6e-8409-4713-becf-e8d89347c853'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:13:00.098826 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:13:00.100019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3989c95816d542ae9f002e2b5e5338a7', 'request_id': 'req-45f78a6e-8409-4713-becf-e8d89347c853'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:13:00.106080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3989c95816d542ae9f002e2b5e5338a7', 'request_id': 'req-45f78a6e-8409-4713-becf-e8d89347c853'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:13:00.188716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] The token cache is being invalidate because project 3989c95816d542ae9f002e2b5e5338a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:13:00.189037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:13:00.189652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-45f78a6e-8409-4713-becf-e8d89347c853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:13:00.191183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1132/2258] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:12:59 2022] DELETE /identity/v3/projects/3989c95816d542ae9f002e2b5e5338a7 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:13:09.493164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:DELETE=10,keystone:SELECT=48,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:13:10.249363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=5,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:13:14.456213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8050c38-4296-4b0e-957e-00f774275fd5 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:13:14.465109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8050c38-4296-4b0e-957e-00f774275fd5 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:13:14.518471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e01d3c-ffa9-418e-91ff-936c9650c321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:13:14.519123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01d3c-ffa9-418e-91ff-936c9650c321 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:13:14.519409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01d3c-ffa9-418e-91ff-936c9650c321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:13:14.519755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e01d3c-ffa9-418e-91ff-936c9650c321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:13:14.521609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b3e01d3c-ffa9-418e-91ff-936c9650c321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:13:14.585248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e01d3c-ffa9-418e-91ff-936c9650c321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:13:14.586677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e01d3c-ffa9-418e-91ff-936c9650c321 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:13:14.596123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1127/2259] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:13:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:13:15.369464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4ca84-1fa0-47a6-a7bf-ff559486a746 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:13:15.369979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4ca84-1fa0-47a6-a7bf-ff559486a746 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:13:15.370071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4ca84-1fa0-47a6-a7bf-ff559486a746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:13:15.388193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-35f4ca84-1fa0-47a6-a7bf-ff559486a746 None None] MFA Rules not processed for user `8b6e665914774eabba120e986e6bb282`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:13:15.389471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-35f4ca84-1fa0-47a6-a7bf-ff559486a746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:13:15.444782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1133/2260] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:13:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:13:24.591880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=50 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:13:25.413677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e262cf-db72-4f3e-9b84-cbca1fcfd0d5 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:13:25.414047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e262cf-db72-4f3e-9b84-cbca1fcfd0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:13:25.414390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e262cf-db72-4f3e-9b84-cbca1fcfd0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:13:25.431910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-48e262cf-db72-4f3e-9b84-cbca1fcfd0d5 None None] MFA Rules not processed for user `cb0474d4e6dd4d2b9918fd0ebe803dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:13:25.433298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-48e262cf-db72-4f3e-9b84-cbca1fcfd0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:13:25.437446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=25 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:13:25.489744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1128/2261] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:13:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:13:35.482708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=25 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:13:44.024661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62545b50-7006-4e3d-b86f-e805ec373804 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:13:44.025163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62545b50-7006-4e3d-b86f-e805ec373804 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:13:44.025163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62545b50-7006-4e3d-b86f-e805ec373804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:13:44.042942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-62545b50-7006-4e3d-b86f-e805ec373804 None None] MFA Rules not processed for user `d7e2fc1ec676438392d7925b43b806f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:13:44.044137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-62545b50-7006-4e3d-b86f-e805ec373804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:13:44.100305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1134/2262] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:13:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:13:44.137005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e262cf-db72-4f3e-9b84-cbca1fcfd0d5 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:13:44.138154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-48e262cf-db72-4f3e-9b84-cbca1fcfd0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:13:44.194419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e262cf-db72-4f3e-9b84-cbca1fcfd0d5 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:13:44.198116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492edb6-63ba-462f-9393-8067a6be4bca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b712bd7d8d4930a3bfc99581be8cb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:13:44.198749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492edb6-63ba-462f-9393-8067a6be4bca service glance] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:13:44.199028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492edb6-63ba-462f-9393-8067a6be4bca service glance] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:13:44.199301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492edb6-63ba-462f-9393-8067a6be4bca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:13:44.201066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5492edb6-63ba-462f-9393-8067a6be4bca service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:13:44.273576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492edb6-63ba-462f-9393-8067a6be4bca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:13:44.275128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492edb6-63ba-462f-9393-8067a6be4bca service glance] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:13:44.286450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1129/2263] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:13:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:13:44.448502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62545b50-7006-4e3d-b86f-e805ec373804 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:13:44.449986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-62545b50-7006-4e3d-b86f-e805ec373804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:13:44.462269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62545b50-7006-4e3d-b86f-e805ec373804 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:13:44.465796 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eded093-9f39-44af-b0ab-78be34188d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:13:44.466436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eded093-9f39-44af-b0ab-78be34188d50 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:13:44.466719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eded093-9f39-44af-b0ab-78be34188d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:13:44.466991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eded093-9f39-44af-b0ab-78be34188d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:13:44.468742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1eded093-9f39-44af-b0ab-78be34188d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:13:44.483207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eded093-9f39-44af-b0ab-78be34188d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:13:44.484922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eded093-9f39-44af-b0ab-78be34188d50 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:13:44.496202 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1135/2264] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:13:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:13:54.281435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=54 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:13:54.490992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=31 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:01.492704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492edb6-63ba-462f-9393-8067a6be4bca service glance] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:01.494121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5492edb6-63ba-462f-9393-8067a6be4bca service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:01.551077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492edb6-63ba-462f-9393-8067a6be4bca service glance] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:01.554799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f238db-d3b3-4045-8980-68dd387eac7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:01.555471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f238db-d3b3-4045-8980-68dd387eac7f service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:01.555753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f238db-d3b3-4045-8980-68dd387eac7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:01.556030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f238db-d3b3-4045-8980-68dd387eac7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:01.557844 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-34f238db-d3b3-4045-8980-68dd387eac7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:01.616357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f238db-d3b3-4045-8980-68dd387eac7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:01.617832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f238db-d3b3-4045-8980-68dd387eac7f service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:01.626985 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1130/2265] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:14:01.627238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eded093-9f39-44af-b0ab-78be34188d50 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:01.628512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1eded093-9f39-44af-b0ab-78be34188d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:01.640850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eded093-9f39-44af-b0ab-78be34188d50 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:01.644407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e5312-91ee-41a2-8acb-26af9a16c7ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:01.645147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e5312-91ee-41a2-8acb-26af9a16c7ac service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:01.645431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e5312-91ee-41a2-8acb-26af9a16c7ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:01.645741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e5312-91ee-41a2-8acb-26af9a16c7ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:01.654846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e5312-91ee-41a2-8acb-26af9a16c7ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:01.656271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e5312-91ee-41a2-8acb-26af9a16c7ac service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:01.665260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1136/2266] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:14:02.029684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f238db-d3b3-4045-8980-68dd387eac7f service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:02.039032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f238db-d3b3-4045-8980-68dd387eac7f service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:02.042658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42926ae5-5467-49bb-8b52-c5c1425367ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:02.043296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42926ae5-5467-49bb-8b52-c5c1425367ed service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:02.043583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42926ae5-5467-49bb-8b52-c5c1425367ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:02.043858 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42926ae5-5467-49bb-8b52-c5c1425367ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:02.045637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-42926ae5-5467-49bb-8b52-c5c1425367ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:02.056254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42926ae5-5467-49bb-8b52-c5c1425367ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:02.057685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42926ae5-5467-49bb-8b52-c5c1425367ed service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:02.066684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1131/2267] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:11.661370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:12.062556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=54 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:16.357204 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c251ba4-464e-4d76-9473-b570a33085f6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:16.357692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c251ba4-464e-4d76-9473-b570a33085f6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:16.357807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c251ba4-464e-4d76-9473-b570a33085f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:16.376162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8c251ba4-464e-4d76-9473-b570a33085f6 None None] MFA Rules not processed for user `0c8512e7c1704fac9cc4fcfe3bf1f18b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:16.377375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8c251ba4-464e-4d76-9473-b570a33085f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:16.384833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1137/2268] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:16.630681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42926ae5-5467-49bb-8b52-c5c1425367ed service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:16.632859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-42926ae5-5467-49bb-8b52-c5c1425367ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:16.650057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42926ae5-5467-49bb-8b52-c5c1425367ed service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:16.655659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e0b45-9da8-43ca-9007-97693ef42107 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:16.656308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e0b45-9da8-43ca-9007-97693ef42107 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:16.656576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e0b45-9da8-43ca-9007-97693ef42107 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:16.656882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e0b45-9da8-43ca-9007-97693ef42107 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:16.665324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-169e0b45-9da8-43ca-9007-97693ef42107 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:16.669834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e0b45-9da8-43ca-9007-97693ef42107 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:16.671444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e0b45-9da8-43ca-9007-97693ef42107 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:16.681383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1132/2269] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:16.688095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c251ba4-464e-4d76-9473-b570a33085f6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:16.698386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c251ba4-464e-4d76-9473-b570a33085f6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:16.702074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e55481-ed95-4328-84ee-83b6b3a29339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:16.702718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e55481-ed95-4328-84ee-83b6b3a29339 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:16.703022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e55481-ed95-4328-84ee-83b6b3a29339 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:16.703283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e55481-ed95-4328-84ee-83b6b3a29339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:16.705071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f0e55481-ed95-4328-84ee-83b6b3a29339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:16.715536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e55481-ed95-4328-84ee-83b6b3a29339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:16.717003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e55481-ed95-4328-84ee-83b6b3a29339 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:16.726359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1138/2270] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:17.152333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e0b45-9da8-43ca-9007-97693ef42107 service nova] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:17.154300 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-169e0b45-9da8-43ca-9007-97693ef42107 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:17.225517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e0b45-9da8-43ca-9007-97693ef42107 service nova] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:17.237459 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5ff33-903d-4a93-9361-756cd8537748 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:17.238222 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5ff33-903d-4a93-9361-756cd8537748 service placement] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:17.238517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5ff33-903d-4a93-9361-756cd8537748 service placement] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:17.238790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5ff33-903d-4a93-9361-756cd8537748 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:17.241282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-98c5ff33-903d-4a93-9361-756cd8537748 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:17.253240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5ff33-903d-4a93-9361-756cd8537748 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:17.254840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5ff33-903d-4a93-9361-756cd8537748 service placement] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:17.265893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1133/2271] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:14:21.728259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e55481-ed95-4328-84ee-83b6b3a29339 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:21.737257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e55481-ed95-4328-84ee-83b6b3a29339 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:21.740859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446bfb2-cd63-4118-b53d-b8dc26078796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:21.741477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446bfb2-cd63-4118-b53d-b8dc26078796 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:21.741761 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446bfb2-cd63-4118-b53d-b8dc26078796 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:21.742082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446bfb2-cd63-4118-b53d-b8dc26078796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:21.743837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6446bfb2-cd63-4118-b53d-b8dc26078796 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:21.753670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446bfb2-cd63-4118-b53d-b8dc26078796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:21.755099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446bfb2-cd63-4118-b53d-b8dc26078796 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:21.763327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1139/2272] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:27.261595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=38 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:29.699812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5ff33-903d-4a93-9361-756cd8537748 service placement] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:29.701281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-98c5ff33-903d-4a93-9361-756cd8537748 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:29.713262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5ff33-903d-4a93-9361-756cd8537748 service placement] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:29.716921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19352066-bfe0-4b84-859e-930cf6c830af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:29.717559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19352066-bfe0-4b84-859e-930cf6c830af service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:29.717842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19352066-bfe0-4b84-859e-930cf6c830af service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:29.718116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19352066-bfe0-4b84-859e-930cf6c830af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:29.719851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-19352066-bfe0-4b84-859e-930cf6c830af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:29.784194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19352066-bfe0-4b84-859e-930cf6c830af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:29.785667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19352066-bfe0-4b84-859e-930cf6c830af service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:29.795717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1134/2273] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:31.759750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=15 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:33.578850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446bfb2-cd63-4118-b53d-b8dc26078796 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:33.580500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6446bfb2-cd63-4118-b53d-b8dc26078796 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:33.597432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446bfb2-cd63-4118-b53d-b8dc26078796 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:33.602925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:33.603950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:33.604300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:33.604645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] PATH_INFO: `/v3/users/8b6e665914774eabba120e986e6bb282` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:33.605963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6e665914774eabba120e986e6bb282)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:33.607997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:33.761584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6e665914774eabba120e986e6bb282'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:33.768659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6e665914774eabba120e986e6bb282'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:33.769427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6e665914774eabba120e986e6bb282'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:33.770030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6e665914774eabba120e986e6bb282'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:33.781936 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6390cb03-a938-43ad-a631-3d7660aecda6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6e665914774eabba120e986e6bb282'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:33.851493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1140/2274] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:14:33 2022] DELETE /identity/v3/users/8b6e665914774eabba120e986e6bb282 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:14:34.145915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19352066-bfe0-4b84-859e-930cf6c830af service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:34.156795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19352066-bfe0-4b84-859e-930cf6c830af service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:34.219211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:34.220119 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:34.220407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:34.220680 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] PATH_INFO: `/v3/projects/bbe6a2de3cba4aa7b365f30f5b2f745c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:34.221890 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbe6a2de3cba4aa7b365f30f5b2f745c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:34.223808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:34.331562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe6a2de3cba4aa7b365f30f5b2f745c', 'request_id': 'req-0dd83e4e-389d-48eb-8ef0-d38852df15d1'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:34.337802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe6a2de3cba4aa7b365f30f5b2f745c', 'request_id': 'req-0dd83e4e-389d-48eb-8ef0-d38852df15d1'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:34.430298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe6a2de3cba4aa7b365f30f5b2f745c', 'request_id': 'req-0dd83e4e-389d-48eb-8ef0-d38852df15d1'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:34.441320 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] The token cache is being invalidate because project bbe6a2de3cba4aa7b365f30f5b2f745c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:14:34.441613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:34.442230 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0dd83e4e-389d-48eb-8ef0-d38852df15d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:34.443759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1135/2275] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:14:34 2022] DELETE /identity/v3/projects/bbe6a2de3cba4aa7b365f30f5b2f745c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:14:34.449210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fda6bd-d9b0-4ef0-b77b-ab1dace38228 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:34.449525 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fda6bd-d9b0-4ef0-b77b-ab1dace38228 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:34.449804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fda6bd-d9b0-4ef0-b77b-ab1dace38228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:34.468321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-89fda6bd-d9b0-4ef0-b77b-ab1dace38228 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:34.471652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-89fda6bd-d9b0-4ef0-b77b-ab1dace38228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:34.532975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1141/2276] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:14:34.541874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586c859-f0cc-4f78-a6f1-ee9276b685a4 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:34.542164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586c859-f0cc-4f78-a6f1-ee9276b685a4 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:34.542440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586c859-f0cc-4f78-a6f1-ee9276b685a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:34.560835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-5586c859-f0cc-4f78-a6f1-ee9276b685a4 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:34.562046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5586c859-f0cc-4f78-a6f1-ee9276b685a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:34.570608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1136/2277] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:34.574830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fda6bd-d9b0-4ef0-b77b-ab1dace38228 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:34.585515 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fda6bd-d9b0-4ef0-b77b-ab1dace38228 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:34.590998 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb86e8-f006-4208-a5ed-2fcee1755a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:34.591648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb86e8-f006-4208-a5ed-2fcee1755a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:34.591929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb86e8-f006-4208-a5ed-2fcee1755a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:34.592205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb86e8-f006-4208-a5ed-2fcee1755a9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:34.592972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb86e8-f006-4208-a5ed-2fcee1755a9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:14:34.593278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb86e8-f006-4208-a5ed-2fcee1755a9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:34.598045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb86e8-f006-4208-a5ed-2fcee1755a9b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:34.606923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1142/2278] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:14:34.611700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586c859-f0cc-4f78-a6f1-ee9276b685a4 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:34.621342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586c859-f0cc-4f78-a6f1-ee9276b685a4 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:34.626827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3d662-580a-46d1-9b70-f490c841b624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:34.627584 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3d662-580a-46d1-9b70-f490c841b624 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:34.627962 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3d662-580a-46d1-9b70-f490c841b624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:34.628211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3d662-580a-46d1-9b70-f490c841b624 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:34.628948 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3d662-580a-46d1-9b70-f490c841b624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:34.630910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3d662-580a-46d1-9b70-f490c841b624 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:34.646223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:34.735947 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1137/2279] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:14:34 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:14:34.737235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb86e8-f006-4208-a5ed-2fcee1755a9b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:34.749365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb86e8-f006-4208-a5ed-2fcee1755a9b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:34.777066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddbab5-f573-48f6-a4e0-fd466cb1a225 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:34.777551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddbab5-f573-48f6-a4e0-fd466cb1a225 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:34.778014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddbab5-f573-48f6-a4e0-fd466cb1a225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:34.799469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-ebddbab5-f573-48f6-a4e0-fd466cb1a225 None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:34.801056 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ebddbab5-f573-48f6-a4e0-fd466cb1a225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:34.818596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ba58f-e608-44b3-a381-f69cfc1bcd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:34.819812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ba58f-e608-44b3-a381-f69cfc1bcd7c admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:34.820257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ba58f-e608-44b3-a381-f69cfc1bcd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:34.820605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ba58f-e608-44b3-a381-f69cfc1bcd7c admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:34.821394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ba58f-e608-44b3-a381-f69cfc1bcd7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:34.823502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ba58f-e608-44b3-a381-f69cfc1bcd7c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:34.830529 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:34.863821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1138/2280] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:14:34.908706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1143/2281] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:14:34 2022] POST /identity/v3/users => generated 302 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:14:34.914704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddbab5-f573-48f6-a4e0-fd466cb1a225 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:34.926906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddbab5-f573-48f6-a4e0-fd466cb1a225 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:34.932108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d449fa-e481-450f-b968-7b56f186a3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:34.932809 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d449fa-e481-450f-b968-7b56f186a3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:34.933181 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d449fa-e481-450f-b968-7b56f186a3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:34.933461 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d449fa-e481-450f-b968-7b56f186a3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:34.934202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d449fa-e481-450f-b968-7b56f186a3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:34.935876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d449fa-e481-450f-b968-7b56f186a3a9 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:34.949866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1139/2282] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:14:34.953931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ba58f-e608-44b3-a381-f69cfc1bcd7c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:34.964594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ba58f-e608-44b3-a381-f69cfc1bcd7c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:34.969306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c60dfe-fbc0-40af-9842-952700338a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:34.969966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c60dfe-fbc0-40af-9842-952700338a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:34.970248 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c60dfe-fbc0-40af-9842-952700338a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:34.970519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c60dfe-fbc0-40af-9842-952700338a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:34.971243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c60dfe-fbc0-40af-9842-952700338a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:34.972862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c60dfe-fbc0-40af-9842-952700338a31 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:34.987041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1144/2283] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:14:34.991740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d449fa-e481-450f-b968-7b56f186a3a9 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:34.999355 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d449fa-e481-450f-b968-7b56f186a3a9 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:35.003973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522617b-63ad-4f15-a54c-ae966d245a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:35.004881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522617b-63ad-4f15-a54c-ae966d245a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:35.005165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522617b-63ad-4f15-a54c-ae966d245a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:35.005440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522617b-63ad-4f15-a54c-ae966d245a3a admin admin] PATH_INFO: `/v3/projects/ba10550b63024bcea9ad172f1f13b4f9/users/3a05a4430b3746ca94319ee12cba335c/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:35.020557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c60dfe-fbc0-40af-9842-952700338a31 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:35.022570 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-08c60dfe-fbc0-40af-9842-952700338a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:35.030311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522617b-63ad-4f15-a54c-ae966d245a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba10550b63024bcea9ad172f1f13b4f9, user_id=3a05a4430b3746ca94319ee12cba335c, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:35.033038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522617b-63ad-4f15-a54c-ae966d245a3a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:35.103150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1140/2284] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:14:34 2022] PUT /identity/v3/projects/ba10550b63024bcea9ad172f1f13b4f9/users/3a05a4430b3746ca94319ee12cba335c/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:14:35.106245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c60dfe-fbc0-40af-9842-952700338a31 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:35.108586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79a11-79e4-4ae6-8f22-95f778db35ff None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:35.112034 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79a11-79e4-4ae6-8f22-95f778db35ff None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:35.112443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79a11-79e4-4ae6-8f22-95f778db35ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:35.132880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-cdd79a11-79e4-4ae6-8f22-95f778db35ff None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:35.134037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-cdd79a11-79e4-4ae6-8f22-95f778db35ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:35.177812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79010e4-ad79-4e73-99e0-c48be7900904 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:35.179008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79010e4-ad79-4e73-99e0-c48be7900904 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:35.179298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79010e4-ad79-4e73-99e0-c48be7900904 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:35.179577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79010e4-ad79-4e73-99e0-c48be7900904 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:35.216117 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1141/2285] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:14:35.226120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79a11-79e4-4ae6-8f22-95f778db35ff None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:35.228168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-cdd79a11-79e4-4ae6-8f22-95f778db35ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:35.257838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79010e4-ad79-4e73-99e0-c48be7900904 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:35.259552 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79010e4-ad79-4e73-99e0-c48be7900904 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:35.269370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1145/2286] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:14:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 05 16:14:35.323806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79a11-79e4-4ae6-8f22-95f778db35ff None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:35.328228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6c528-1c96-4839-82d0-cf652ccd972d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:35.328941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6c528-1c96-4839-82d0-cf652ccd972d service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:35.329219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6c528-1c96-4839-82d0-cf652ccd972d service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:35.330443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6c528-1c96-4839-82d0-cf652ccd972d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:35.344385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6c528-1c96-4839-82d0-cf652ccd972d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:35.345867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6c528-1c96-4839-82d0-cf652ccd972d service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:35.359496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1142/2287] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:36.642638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79010e4-ad79-4e73-99e0-c48be7900904 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:36.644608 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d79010e4-ad79-4e73-99e0-c48be7900904 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:36.696312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:36.757027 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79010e4-ad79-4e73-99e0-c48be7900904 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:36.763106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7421f-72e1-4da0-b14a-220339f70415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:36.766782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7421f-72e1-4da0-b14a-220339f70415 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:36.766782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7421f-72e1-4da0-b14a-220339f70415 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:36.766782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7421f-72e1-4da0-b14a-220339f70415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:36.766782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dde7421f-72e1-4da0-b14a-220339f70415 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:36.780235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7421f-72e1-4da0-b14a-220339f70415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:36.781860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7421f-72e1-4da0-b14a-220339f70415 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:36.801209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1146/2288] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:36.804121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f073f-bb86-496e-b752-a7d477bbbf11 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:36.804593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f073f-bb86-496e-b752-a7d477bbbf11 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:36.804699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f073f-bb86-496e-b752-a7d477bbbf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:36.831845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:36.838107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d00f073f-bb86-496e-b752-a7d477bbbf11 None None] MFA Rules not processed for user `3a05a4430b3746ca94319ee12cba335c`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:36.838789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d00f073f-bb86-496e-b752-a7d477bbbf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:36.960596 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1143/2289] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:14:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:36.962332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7421f-72e1-4da0-b14a-220339f70415 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:36.973402 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7421f-72e1-4da0-b14a-220339f70415 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:36.980742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c76e2-5818-4321-9a40-938b9af081c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:36.980742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c76e2-5818-4321-9a40-938b9af081c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:36.980742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c76e2-5818-4321-9a40-938b9af081c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:36.980742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c76e2-5818-4321-9a40-938b9af081c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:36.981225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c76e2-5818-4321-9a40-938b9af081c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:36.983288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c76e2-5818-4321-9a40-938b9af081c3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:37.075184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1147/2290] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:14:36 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:14:37.076482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f073f-bb86-496e-b752-a7d477bbbf11 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:37.100683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f073f-bb86-496e-b752-a7d477bbbf11 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:37.200490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f08b74-91e0-4b50-ad5d-519cc22228aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:37.201385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f08b74-91e0-4b50-ad5d-519cc22228aa admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:37.201773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f08b74-91e0-4b50-ad5d-519cc22228aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:37.202198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f08b74-91e0-4b50-ad5d-519cc22228aa admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:37.203045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f08b74-91e0-4b50-ad5d-519cc22228aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:37.205376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f08b74-91e0-4b50-ad5d-519cc22228aa admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:37.324903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1144/2291] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:14:37 2022] POST /identity/v3/users => generated 308 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:14:37.326091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c76e2-5818-4321-9a40-938b9af081c3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:37.340955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c76e2-5818-4321-9a40-938b9af081c3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:37.353802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1a330-cf46-4c5e-a256-5995436ff797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:37.354394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1a330-cf46-4c5e-a256-5995436ff797 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:37.354693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1a330-cf46-4c5e-a256-5995436ff797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:37.354955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1a330-cf46-4c5e-a256-5995436ff797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:37.355841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1a330-cf46-4c5e-a256-5995436ff797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:37.359458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1a330-cf46-4c5e-a256-5995436ff797 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:37.380822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1148/2292] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:14:37.382673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f08b74-91e0-4b50-ad5d-519cc22228aa admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:37.394476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f08b74-91e0-4b50-ad5d-519cc22228aa admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:37.401111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6a981-3864-45b1-8e34-f8eaaa989918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:37.402187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6a981-3864-45b1-8e34-f8eaaa989918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:37.402761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6a981-3864-45b1-8e34-f8eaaa989918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:37.403189 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6a981-3864-45b1-8e34-f8eaaa989918 admin admin] PATH_INFO: `/v3/projects/c584066616ac4ac79cbf867ea08b52e0/users/67aeb9e072e14f28a04da396f0a2804f/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:37.436450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6a981-3864-45b1-8e34-f8eaaa989918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c584066616ac4ac79cbf867ea08b52e0, user_id=67aeb9e072e14f28a04da396f0a2804f, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:37.439525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6a981-3864-45b1-8e34-f8eaaa989918 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:37.519990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1145/2293] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:14:37 2022] PUT /identity/v3/projects/c584066616ac4ac79cbf867ea08b52e0/users/67aeb9e072e14f28a04da396f0a2804f/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:14:37.524326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1a330-cf46-4c5e-a256-5995436ff797 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:37.535108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1a330-cf46-4c5e-a256-5995436ff797 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:37.636542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e469f0-1a50-4d66-a607-b7aec0e994c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:37.637542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e469f0-1a50-4d66-a607-b7aec0e994c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:37.639947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e469f0-1a50-4d66-a607-b7aec0e994c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:37.639947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e469f0-1a50-4d66-a607-b7aec0e994c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:37.639947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e469f0-1a50-4d66-a607-b7aec0e994c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:37.641119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e469f0-1a50-4d66-a607-b7aec0e994c3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:37.655067 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1149/2294] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:14:37.659038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6a981-3864-45b1-8e34-f8eaaa989918 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:37.670253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6a981-3864-45b1-8e34-f8eaaa989918 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:37.676952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5f767-6abe-4fec-b3e8-e7b72c357e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:37.678452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5f767-6abe-4fec-b3e8-e7b72c357e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:37.678956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5f767-6abe-4fec-b3e8-e7b72c357e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:37.679433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5f767-6abe-4fec-b3e8-e7b72c357e8e admin admin] PATH_INFO: `/v3/projects/c584066616ac4ac79cbf867ea08b52e0/users/67aeb9e072e14f28a04da396f0a2804f/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:37.684002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5f767-6abe-4fec-b3e8-e7b72c357e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c584066616ac4ac79cbf867ea08b52e0, user_id=67aeb9e072e14f28a04da396f0a2804f, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:37.687079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5f767-6abe-4fec-b3e8-e7b72c357e8e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:37.704413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1146/2295] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:14:37 2022] PUT /identity/v3/projects/c584066616ac4ac79cbf867ea08b52e0/users/67aeb9e072e14f28a04da396f0a2804f/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:14:39.743295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941848ec-100c-40cf-8e64-9029106fa53d None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:39.744107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941848ec-100c-40cf-8e64-9029106fa53d None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:39.744107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941848ec-100c-40cf-8e64-9029106fa53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:39.789587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-941848ec-100c-40cf-8e64-9029106fa53d None None] MFA Rules not processed for user `67aeb9e072e14f28a04da396f0a2804f`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:39.791217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-941848ec-100c-40cf-8e64-9029106fa53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:39.893765 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:39.911366 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1150/2296] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:14:39 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:41.426835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5f767-6abe-4fec-b3e8-e7b72c357e8e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:41.535009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:41.546124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5f767-6abe-4fec-b3e8-e7b72c357e8e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:41.552343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:41.555074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:41.555074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:41.555074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] PATH_INFO: `/v3/users/3a05a4430b3746ca94319ee12cba335c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:41.555407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a05a4430b3746ca94319ee12cba335c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:41.557410 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:42.073654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a05a4430b3746ca94319ee12cba335c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:42.080989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a05a4430b3746ca94319ee12cba335c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:42.081692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a05a4430b3746ca94319ee12cba335c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:42.093667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a05a4430b3746ca94319ee12cba335c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:42.094318 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a05a4430b3746ca94319ee12cba335c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:42.184527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1147/2297] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:14:41 2022] DELETE /identity/v3/users/3a05a4430b3746ca94319ee12cba335c => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:14:42.486744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941848ec-100c-40cf-8e64-9029106fa53d None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:42.504116 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941848ec-100c-40cf-8e64-9029106fa53d None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:42.653254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:42.654207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:42.654523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:42.654820 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] PATH_INFO: `/v3/projects/ba10550b63024bcea9ad172f1f13b4f9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:42.657034 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba10550b63024bcea9ad172f1f13b4f9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:42.659345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:42.809247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba10550b63024bcea9ad172f1f13b4f9', 'request_id': 'req-169d976b-2a61-4d08-aa09-659e5510f64d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:42.823827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba10550b63024bcea9ad172f1f13b4f9', 'request_id': 'req-169d976b-2a61-4d08-aa09-659e5510f64d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:42.835076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba10550b63024bcea9ad172f1f13b4f9', 'request_id': 'req-169d976b-2a61-4d08-aa09-659e5510f64d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:42.934412 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] The token cache is being invalidate because project ba10550b63024bcea9ad172f1f13b4f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:14:42.934705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:42.935370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:42.940095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1151/2298] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:14:42 2022] DELETE /identity/v3/projects/ba10550b63024bcea9ad172f1f13b4f9 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:14:42.941215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:42.942734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:42.960629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416067e-a0b6-427e-92f9-ca0853c223bb admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:43.032782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:43.033986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:43.034802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:43.035316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] PATH_INFO: `/v3/users/67aeb9e072e14f28a04da396f0a2804f` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:43.036609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] RBAC: Authorizing `identity:delete_user(user_id=67aeb9e072e14f28a04da396f0a2804f)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:43.038856 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:43.226332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67aeb9e072e14f28a04da396f0a2804f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:43.234989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67aeb9e072e14f28a04da396f0a2804f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:43.235692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67aeb9e072e14f28a04da396f0a2804f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:43.252439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:43.257539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67aeb9e072e14f28a04da396f0a2804f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:43.258421 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-39496c87-385a-4b42-8e5d-c3ff5233640a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67aeb9e072e14f28a04da396f0a2804f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:43.353133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1148/2299] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:14:42 2022] DELETE /identity/v3/users/67aeb9e072e14f28a04da396f0a2804f => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:14:43.630317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:43.633065 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:43.661161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d976b-2a61-4d08-aa09-659e5510f64d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:43.730711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:43.731587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:43.731878 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:43.732152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] PATH_INFO: `/v3/projects/c584066616ac4ac79cbf867ea08b52e0` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:43.733388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c584066616ac4ac79cbf867ea08b52e0)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:43.737524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:43.863790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:43.894690 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c584066616ac4ac79cbf867ea08b52e0', 'request_id': 'req-fe500064-62c1-479c-be77-854d4b37d219'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:43.905875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c584066616ac4ac79cbf867ea08b52e0', 'request_id': 'req-fe500064-62c1-479c-be77-854d4b37d219'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:43.913071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c584066616ac4ac79cbf867ea08b52e0', 'request_id': 'req-fe500064-62c1-479c-be77-854d4b37d219'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:43.995493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] The token cache is being invalidate because project c584066616ac4ac79cbf867ea08b52e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:14:43.998809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:43.998809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-fe500064-62c1-479c-be77-854d4b37d219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:14:44.000000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1152/2300] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:14:43 2022] DELETE /identity/v3/projects/c584066616ac4ac79cbf867ea08b52e0 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:14:44.006064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399c5f9-40ea-464a-8059-6d46b57fab64 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:44.007295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399c5f9-40ea-464a-8059-6d46b57fab64 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:44.007295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399c5f9-40ea-464a-8059-6d46b57fab64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:44.037917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3399c5f9-40ea-464a-8059-6d46b57fab64 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:44.039321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3399c5f9-40ea-464a-8059-6d46b57fab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:44.137328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1149/2301] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:44.140558 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad00bc6-16cd-4e76-9dd9-eafff9e21bfe None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:44.145087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad00bc6-16cd-4e76-9dd9-eafff9e21bfe None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:44.145480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad00bc6-16cd-4e76-9dd9-eafff9e21bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:44.167635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6ad00bc6-16cd-4e76-9dd9-eafff9e21bfe None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:44.169028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6ad00bc6-16cd-4e76-9dd9-eafff9e21bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:44.180434 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1153/2302] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:44.185296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399c5f9-40ea-464a-8059-6d46b57fab64 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:44.196782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399c5f9-40ea-464a-8059-6d46b57fab64 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:44.207904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4134faf-7cba-4967-9dba-855b35bc4d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:44.208595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4134faf-7cba-4967-9dba-855b35bc4d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:44.208909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4134faf-7cba-4967-9dba-855b35bc4d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:44.209704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4134faf-7cba-4967-9dba-855b35bc4d01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:44.210643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4134faf-7cba-4967-9dba-855b35bc4d01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:14:44.210964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4134faf-7cba-4967-9dba-855b35bc4d01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:44.212586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4134faf-7cba-4967-9dba-855b35bc4d01 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:44.226638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1150/2303] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:14:44.230401 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad00bc6-16cd-4e76-9dd9-eafff9e21bfe None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:44.241082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad00bc6-16cd-4e76-9dd9-eafff9e21bfe None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:44.245883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51137652-f4e1-45a8-b939-da5a4b1c0aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:44.246548 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51137652-f4e1-45a8-b939-da5a4b1c0aef admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:44.246833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51137652-f4e1-45a8-b939-da5a4b1c0aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:44.247109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51137652-f4e1-45a8-b939-da5a4b1c0aef admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:44.247813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51137652-f4e1-45a8-b939-da5a4b1c0aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:44.249795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51137652-f4e1-45a8-b939-da5a4b1c0aef admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:44.344061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1154/2304] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:14:44 2022] POST /identity/v3/projects => generated 355 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:14:44.348237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4134faf-7cba-4967-9dba-855b35bc4d01 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:44.363363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4134faf-7cba-4967-9dba-855b35bc4d01 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:44.445775 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a928e09-38dd-48e3-9383-7c3814eb77ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:44.446582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a928e09-38dd-48e3-9383-7c3814eb77ae admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:44.449557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a928e09-38dd-48e3-9383-7c3814eb77ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:44.449557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a928e09-38dd-48e3-9383-7c3814eb77ae admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:44.449557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a928e09-38dd-48e3-9383-7c3814eb77ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:44.451826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a928e09-38dd-48e3-9383-7c3814eb77ae admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:44.532861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1151/2305] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:14:44 2022] POST /identity/v3/users => generated 278 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:14:44.540104 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51137652-f4e1-45a8-b939-da5a4b1c0aef admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:44.552207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51137652-f4e1-45a8-b939-da5a4b1c0aef admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:44.562085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73d11b-e7f2-4437-a7dc-0eb56e7bcf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:44.563789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73d11b-e7f2-4437-a7dc-0eb56e7bcf59 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:44.563789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73d11b-e7f2-4437-a7dc-0eb56e7bcf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:44.563789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73d11b-e7f2-4437-a7dc-0eb56e7bcf59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:44.564647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73d11b-e7f2-4437-a7dc-0eb56e7bcf59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:44.566365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73d11b-e7f2-4437-a7dc-0eb56e7bcf59 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:44.583352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1155/2306] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:14:44.587243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a928e09-38dd-48e3-9383-7c3814eb77ae admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:44.602362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a928e09-38dd-48e3-9383-7c3814eb77ae admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:44.611270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489edd2c-34ac-407f-b405-7ae6b6d49879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:44.612232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489edd2c-34ac-407f-b405-7ae6b6d49879 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:44.612694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489edd2c-34ac-407f-b405-7ae6b6d49879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:44.613167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489edd2c-34ac-407f-b405-7ae6b6d49879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:44.614247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489edd2c-34ac-407f-b405-7ae6b6d49879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:44.616363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489edd2c-34ac-407f-b405-7ae6b6d49879 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:44.629363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1152/2307] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:14:44.633791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73d11b-e7f2-4437-a7dc-0eb56e7bcf59 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:44.648386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73d11b-e7f2-4437-a7dc-0eb56e7bcf59 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:44.656803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73017c-dcbe-485b-af1a-6e8a8084df7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:44.657840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73017c-dcbe-485b-af1a-6e8a8084df7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:44.658128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73017c-dcbe-485b-af1a-6e8a8084df7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:44.658430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73017c-dcbe-485b-af1a-6e8a8084df7e admin admin] PATH_INFO: `/v3/projects/fed87ad05186438181094010f1abd7e7/users/a84974cd49eb4902a81df688722acb78/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:44.677077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73017c-dcbe-485b-af1a-6e8a8084df7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fed87ad05186438181094010f1abd7e7, user_id=a84974cd49eb4902a81df688722acb78, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:44.680808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73017c-dcbe-485b-af1a-6e8a8084df7e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:44.767934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1156/2308] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:14:44 2022] PUT /identity/v3/projects/fed87ad05186438181094010f1abd7e7/users/a84974cd49eb4902a81df688722acb78/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:14:44.770735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36772d-4a89-4a4c-ad14-8bf45652c756 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:44.770735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36772d-4a89-4a4c-ad14-8bf45652c756 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:44.770735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36772d-4a89-4a4c-ad14-8bf45652c756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:44.872900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-fb36772d-4a89-4a4c-ad14-8bf45652c756 None None] MFA Rules not processed for user `a84974cd49eb4902a81df688722acb78`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:44.872900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fb36772d-4a89-4a4c-ad14-8bf45652c756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:44.974725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:45.018835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73017c-dcbe-485b-af1a-6e8a8084df7e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:45.031478 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1153/2309] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:14:44 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:14:45.126164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73017c-dcbe-485b-af1a-6e8a8084df7e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:45.132944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38997807-f244-495e-93db-bc8a0a0b6315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:45.133808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38997807-f244-495e-93db-bc8a0a0b6315 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:45.134371 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38997807-f244-495e-93db-bc8a0a0b6315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:45.134750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38997807-f244-495e-93db-bc8a0a0b6315 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:45.135604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38997807-f244-495e-93db-bc8a0a0b6315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:45.137711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38997807-f244-495e-93db-bc8a0a0b6315 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:45.569180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1157/2310] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:14:45 2022] POST /identity/v3/projects => generated 357 bytes in 551 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:14:45.573942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36772d-4a89-4a4c-ad14-8bf45652c756 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:45.587207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36772d-4a89-4a4c-ad14-8bf45652c756 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:45.672929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe0a4a-81ce-4351-a99f-b44b674b1b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:45.674391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe0a4a-81ce-4351-a99f-b44b674b1b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:45.674391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe0a4a-81ce-4351-a99f-b44b674b1b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:45.674391 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe0a4a-81ce-4351-a99f-b44b674b1b1d admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:45.675201 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe0a4a-81ce-4351-a99f-b44b674b1b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:45.677303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe0a4a-81ce-4351-a99f-b44b674b1b1d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:45.800405 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38997807-f244-495e-93db-bc8a0a0b6315 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:45.810468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1154/2311] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:14:45 2022] POST /identity/v3/users => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:14:45.815869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38997807-f244-495e-93db-bc8a0a0b6315 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:45.822744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cce762-6e8c-40f8-abc8-a9dd576a4a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:45.824207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce762-6e8c-40f8-abc8-a9dd576a4a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:45.824561 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce762-6e8c-40f8-abc8-a9dd576a4a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:45.824947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce762-6e8c-40f8-abc8-a9dd576a4a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:45.826277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cce762-6e8c-40f8-abc8-a9dd576a4a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:45.828095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cce762-6e8c-40f8-abc8-a9dd576a4a22 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:45.843929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe0a4a-81ce-4351-a99f-b44b674b1b1d admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:45.844635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1158/2312] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:14:45.856704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe0a4a-81ce-4351-a99f-b44b674b1b1d admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:45.862557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d8429-37c7-480e-aaa7-66acad980484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:45.864444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d8429-37c7-480e-aaa7-66acad980484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:45.865241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d8429-37c7-480e-aaa7-66acad980484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:45.866544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d8429-37c7-480e-aaa7-66acad980484 admin admin] PATH_INFO: `/v3/projects/eff1faee2cc64482a8fa3b54f391c0b3/users/54142fcb4444418398bc1bcfe2aeaa09/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:45.888811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d8429-37c7-480e-aaa7-66acad980484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff1faee2cc64482a8fa3b54f391c0b3, user_id=54142fcb4444418398bc1bcfe2aeaa09, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:45.891986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d8429-37c7-480e-aaa7-66acad980484 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:46.006830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1155/2313] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:14:45 2022] PUT /identity/v3/projects/eff1faee2cc64482a8fa3b54f391c0b3/users/54142fcb4444418398bc1bcfe2aeaa09/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:14:46.011360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cce762-6e8c-40f8-abc8-a9dd576a4a22 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:46.026873 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cce762-6e8c-40f8-abc8-a9dd576a4a22 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:46.065399 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:46.121360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79a924-716b-4cdb-b056-f60287b876bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:46.122878 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79a924-716b-4cdb-b056-f60287b876bb admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:46.123334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79a924-716b-4cdb-b056-f60287b876bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:46.124025 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79a924-716b-4cdb-b056-f60287b876bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:46.124979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79a924-716b-4cdb-b056-f60287b876bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:46.126958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79a924-716b-4cdb-b056-f60287b876bb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:46.144201 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d8429-37c7-480e-aaa7-66acad980484 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:46.144989 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1159/2314] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:14:46.165490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d8429-37c7-480e-aaa7-66acad980484 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:46.176603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d115d14-6174-4d84-ba64-4fe096d1c09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:46.178527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d115d14-6174-4d84-ba64-4fe096d1c09c admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:46.178875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d115d14-6174-4d84-ba64-4fe096d1c09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:46.179215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d115d14-6174-4d84-ba64-4fe096d1c09c admin admin] PATH_INFO: `/v3/projects/eff1faee2cc64482a8fa3b54f391c0b3/users/54142fcb4444418398bc1bcfe2aeaa09/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:46.181604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d115d14-6174-4d84-ba64-4fe096d1c09c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff1faee2cc64482a8fa3b54f391c0b3, user_id=54142fcb4444418398bc1bcfe2aeaa09, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:46.184617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d115d14-6174-4d84-ba64-4fe096d1c09c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:46.198966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1156/2315] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:14:46 2022] PUT /identity/v3/projects/eff1faee2cc64482a8fa3b54f391c0b3/users/54142fcb4444418398bc1bcfe2aeaa09/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:14:46.210513 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bf802-90b2-408b-88d8-93db97d2d848 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:46.210871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bf802-90b2-408b-88d8-93db97d2d848 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:46.211411 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bf802-90b2-408b-88d8-93db97d2d848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:46.238796 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d21bf802-90b2-408b-88d8-93db97d2d848 None None] MFA Rules not processed for user `54142fcb4444418398bc1bcfe2aeaa09`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:46.240213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d21bf802-90b2-408b-88d8-93db97d2d848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:46.360815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1160/2316] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:46.370915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d115d14-6174-4d84-ba64-4fe096d1c09c admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:46.372950 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8d115d14-6174-4d84-ba64-4fe096d1c09c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:46.451791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d115d14-6174-4d84-ba64-4fe096d1c09c admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:46.451791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d4ccf-1975-4dce-b198-0f54f056c19c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:46.451791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d4ccf-1975-4dce-b198-0f54f056c19c service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:46.451791 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d4ccf-1975-4dce-b198-0f54f056c19c service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:46.452428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d4ccf-1975-4dce-b198-0f54f056c19c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:46.461925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d4ccf-1975-4dce-b198-0f54f056c19c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:46.464117 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d4ccf-1975-4dce-b198-0f54f056c19c service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:46.476105 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1157/2317] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:46 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:46.725368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bf802-90b2-408b-88d8-93db97d2d848 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:46.736076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bf802-90b2-408b-88d8-93db97d2d848 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:46.741870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1eb996-1497-4073-9eea-f9ac951cbec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:46.742784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eb996-1497-4073-9eea-f9ac951cbec2 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:46.743262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eb996-1497-4073-9eea-f9ac951cbec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:46.743844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eb996-1497-4073-9eea-f9ac951cbec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:46.775120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d4ccf-1975-4dce-b198-0f54f056c19c service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:46.778272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e33d4ccf-1975-4dce-b198-0f54f056c19c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:46.840149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1eb996-1497-4073-9eea-f9ac951cbec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:46.842867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1eb996-1497-4073-9eea-f9ac951cbec2 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:46.846902 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:46.864943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1161/2318] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:46 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:46.910153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d4ccf-1975-4dce-b198-0f54f056c19c service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:46.919067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa815f0-c2ea-4f42-a33a-78ac9bdf00cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:46.919785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa815f0-c2ea-4f42-a33a-78ac9bdf00cb service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:46.920150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa815f0-c2ea-4f42-a33a-78ac9bdf00cb service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:46.920497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa815f0-c2ea-4f42-a33a-78ac9bdf00cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:46.922943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5fa815f0-c2ea-4f42-a33a-78ac9bdf00cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:46.940100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa815f0-c2ea-4f42-a33a-78ac9bdf00cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:46.942226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa815f0-c2ea-4f42-a33a-78ac9bdf00cb service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:46.956520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1158/2319] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:50.077839 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1eb996-1497-4073-9eea-f9ac951cbec2 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:50.079648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9b1eb996-1497-4073-9eea-f9ac951cbec2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:50.097109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1eb996-1497-4073-9eea-f9ac951cbec2 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:50.103568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141e6a8-84f7-4384-9707-1b0cc3f5959e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:50.104183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e6a8-84f7-4384-9707-1b0cc3f5959e service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:50.104676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e6a8-84f7-4384-9707-1b0cc3f5959e service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:50.105134 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e6a8-84f7-4384-9707-1b0cc3f5959e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:50.113026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4141e6a8-84f7-4384-9707-1b0cc3f5959e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:50.121655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141e6a8-84f7-4384-9707-1b0cc3f5959e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:50.123256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141e6a8-84f7-4384-9707-1b0cc3f5959e service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:50.134101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1162/2320] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:53.454633 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cc6e4-d9e2-42b8-9bfd-19a38e06003b None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:53.457671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cc6e4-d9e2-42b8-9bfd-19a38e06003b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:53.457671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cc6e4-d9e2-42b8-9bfd-19a38e06003b None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:53.457671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1159/2321] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:14:53.474823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6899a-fca3-4fe0-94dc-39df4c54c417 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:53.475295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6899a-fca3-4fe0-94dc-39df4c54c417 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:53.475492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6899a-fca3-4fe0-94dc-39df4c54c417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:53.497522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-fae6899a-fca3-4fe0-94dc-39df4c54c417 None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:53.498748 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fae6899a-fca3-4fe0-94dc-39df4c54c417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:53.509271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1163/2322] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:14:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:53.521081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cc6e4-d9e2-42b8-9bfd-19a38e06003b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:53.522895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ba5cc6e4-d9e2-42b8-9bfd-19a38e06003b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:53.598006 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cc6e4-d9e2-42b8-9bfd-19a38e06003b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:53.603972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2be75-7513-4f00-ae83-9c2f1610534b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b712bd7d8d4930a3bfc99581be8cb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:53.604759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2be75-7513-4f00-ae83-9c2f1610534b service glance] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:53.605068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2be75-7513-4f00-ae83-9c2f1610534b service glance] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:53.605349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2be75-7513-4f00-ae83-9c2f1610534b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:53.617563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2be75-7513-4f00-ae83-9c2f1610534b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:53.619095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2be75-7513-4f00-ae83-9c2f1610534b service glance] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:53.632150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1160/2323] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:53.848059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346eb3dd-cb39-433b-b535-2fd8aaa1cf1c None None] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:53.848353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346eb3dd-cb39-433b-b535-2fd8aaa1cf1c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:53.848641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346eb3dd-cb39-433b-b535-2fd8aaa1cf1c None None] PATH_INFO: `/` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:53.850219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1164/2324] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:14:53.856254 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69facf59-6265-44ed-bf1c-325dc157734a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:53.856802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69facf59-6265-44ed-bf1c-325dc157734a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:53.857207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69facf59-6265-44ed-bf1c-325dc157734a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:53.875883 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-69facf59-6265-44ed-bf1c-325dc157734a None None] MFA Rules not processed for user `887d2ad3cbf7459a8a59e7442ba9744e`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:53.877110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-69facf59-6265-44ed-bf1c-325dc157734a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:53.928941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1161/2325] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:14:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:53.940426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346eb3dd-cb39-433b-b535-2fd8aaa1cf1c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:53.942152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-346eb3dd-cb39-433b-b535-2fd8aaa1cf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:53.955716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346eb3dd-cb39-433b-b535-2fd8aaa1cf1c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:53.959833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f6dc-e53c-4f22-8df3-e854a3e0934d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:53.960512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f6dc-e53c-4f22-8df3-e854a3e0934d service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:53.960820 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f6dc-e53c-4f22-8df3-e854a3e0934d service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:53.961121 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f6dc-e53c-4f22-8df3-e854a3e0934d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:53.969070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458f6dc-e53c-4f22-8df3-e854a3e0934d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:53.970516 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458f6dc-e53c-4f22-8df3-e854a3e0934d service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:53.979984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1165/2326] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:54.070198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec5c4d-b2fc-4157-a1cb-8e2b884dc868 None None] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:54.070524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec5c4d-b2fc-4157-a1cb-8e2b884dc868 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:54.070828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec5c4d-b2fc-4157-a1cb-8e2b884dc868 None None] PATH_INFO: `/` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:54.072367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1162/2327] 10.222.0.45 () {54 vars in 921 bytes} [Tue Jul 5 16:14:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 16:14:54.079003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a5c1-1f00-4544-bec8-b6bd25f92b55 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:54.081462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a5c1-1f00-4544-bec8-b6bd25f92b55 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:54.081462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a5c1-1f00-4544-bec8-b6bd25f92b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:54.098049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b119a5c1-1f00-4544-bec8-b6bd25f92b55 None None] MFA Rules not processed for user `ef95ab7b924d456195a8761e894d911b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:54.099800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b119a5c1-1f00-4544-bec8-b6bd25f92b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:54.109083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1166/2328] 10.222.0.45 () {58 vars in 1018 bytes} [Tue Jul 5 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:54.120433 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec5c4d-b2fc-4157-a1cb-8e2b884dc868 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:54.131749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec5c4d-b2fc-4157-a1cb-8e2b884dc868 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:54.135734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3334d1-de4c-4f13-b5d6-96486d0e9f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:54.136993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3334d1-de4c-4f13-b5d6-96486d0e9f3e service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:54.137291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3334d1-de4c-4f13-b5d6-96486d0e9f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:54.137594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3334d1-de4c-4f13-b5d6-96486d0e9f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:54.151261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3334d1-de4c-4f13-b5d6-96486d0e9f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:54.152679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3334d1-de4c-4f13-b5d6-96486d0e9f3e service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:54.163969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1163/2329] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:14:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:14:54.843890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47ae47-cada-4b98-8e66-471b44da12e1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:54.844361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47ae47-cada-4b98-8e66-471b44da12e1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:54.844572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47ae47-cada-4b98-8e66-471b44da12e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:54.857319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:54.863197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ec47ae47-cada-4b98-8e66-471b44da12e1 None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:54.864395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ec47ae47-cada-4b98-8e66-471b44da12e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:54.918396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1167/2330] 10.222.0.45 () {58 vars in 967 bytes} [Tue Jul 5 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:54.934450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3334d1-de4c-4f13-b5d6-96486d0e9f3e service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:54.936702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7d3334d1-de4c-4f13-b5d6-96486d0e9f3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:55.014689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3334d1-de4c-4f13-b5d6-96486d0e9f3e service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:55.020073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b451b5-31ce-4f9a-a150-d581f00f5edb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:55.021224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b451b5-31ce-4f9a-a150-d581f00f5edb service swift] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:55.022038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b451b5-31ce-4f9a-a150-d581f00f5edb service swift] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:55.022432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b451b5-31ce-4f9a-a150-d581f00f5edb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:55.027653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:14:55.031564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b451b5-31ce-4f9a-a150-d581f00f5edb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:55.033279 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b451b5-31ce-4f9a-a150-d581f00f5edb service swift] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:55.042216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1164/2331] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:14:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:14:56.473692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5485f53-4c11-4d75-bb48-95d747b860cb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:56.474783 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5485f53-4c11-4d75-bb48-95d747b860cb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:56.474989 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5485f53-4c11-4d75-bb48-95d747b860cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:56.494739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d5485f53-4c11-4d75-bb48-95d747b860cb None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:14:56.496000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d5485f53-4c11-4d75-bb48-95d747b860cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:14:56.503858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1168/2332] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:14:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:14:56.519075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b451b5-31ce-4f9a-a150-d581f00f5edb service swift] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:14:56.529414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b451b5-31ce-4f9a-a150-d581f00f5edb service swift] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:14:56.535597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd1d28-4db8-467a-ae91-06db6f01e6a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:14:56.536248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd1d28-4db8-467a-ae91-06db6f01e6a0 service swift] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:14:56.536533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd1d28-4db8-467a-ae91-06db6f01e6a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:14:56.537338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd1d28-4db8-467a-ae91-06db6f01e6a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:14:56.547262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd1d28-4db8-467a-ae91-06db6f01e6a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:14:56.549005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd1d28-4db8-467a-ae91-06db6f01e6a0 service swift] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:14:56.559187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1165/2333] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:14:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:15:00.262804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5485f53-4c11-4d75-bb48-95d747b860cb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:00.285810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5485f53-4c11-4d75-bb48-95d747b860cb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:00.298340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f053c05-80ed-4e64-aa8f-0b32205f91b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:00.299087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f053c05-80ed-4e64-aa8f-0b32205f91b5 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:00.299448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f053c05-80ed-4e64-aa8f-0b32205f91b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:00.299933 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f053c05-80ed-4e64-aa8f-0b32205f91b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:00.301890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5f053c05-80ed-4e64-aa8f-0b32205f91b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:00.430938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f053c05-80ed-4e64-aa8f-0b32205f91b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:00.432787 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f053c05-80ed-4e64-aa8f-0b32205f91b5 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:00.465542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1169/2334] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:15:01.342465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd1d28-4db8-467a-ae91-06db6f01e6a0 service swift] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:01.344047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-71fd1d28-4db8-467a-ae91-06db6f01e6a0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:01.465786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd1d28-4db8-467a-ae91-06db6f01e6a0 service swift] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:01.471915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e09399-40a8-403f-971b-d437f5dd35ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:01.473392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e09399-40a8-403f-971b-d437f5dd35ad service placement] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:01.473788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e09399-40a8-403f-971b-d437f5dd35ad service placement] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:01.474741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e09399-40a8-403f-971b-d437f5dd35ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:01.476502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-34e09399-40a8-403f-971b-d437f5dd35ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:01.493220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e09399-40a8-403f-971b-d437f5dd35ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:01.495399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e09399-40a8-403f-971b-d437f5dd35ad service placement] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:01.509099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1166/2335] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:15:06.266199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f053c05-80ed-4e64-aa8f-0b32205f91b5 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:06.283832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f053c05-80ed-4e64-aa8f-0b32205f91b5 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:06.295109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf02d8b-2de3-44d9-a135-aa4544c87efd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:06.295843 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf02d8b-2de3-44d9-a135-aa4544c87efd service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:06.296129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf02d8b-2de3-44d9-a135-aa4544c87efd service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:06.296404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf02d8b-2de3-44d9-a135-aa4544c87efd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:06.298315 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3bf02d8b-2de3-44d9-a135-aa4544c87efd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:06.311460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf02d8b-2de3-44d9-a135-aa4544c87efd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:06.313107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf02d8b-2de3-44d9-a135-aa4544c87efd service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:06.323265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1170/2336] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:15:06.376191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e09399-40a8-403f-971b-d437f5dd35ad service placement] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:06.377656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-34e09399-40a8-403f-971b-d437f5dd35ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:06.452895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e09399-40a8-403f-971b-d437f5dd35ad service placement] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:06.457569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:06.458980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:06.459266 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:06.459542 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] PATH_INFO: `/v3/users/80de40d2a02343858849d481590aa570` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:06.461188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80de40d2a02343858849d481590aa570)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:06.463136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:06.724881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80de40d2a02343858849d481590aa570'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:06.733460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80de40d2a02343858849d481590aa570'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:06.735721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80de40d2a02343858849d481590aa570'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:06.746131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:06.748894 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80de40d2a02343858849d481590aa570'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:06.749615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80de40d2a02343858849d481590aa570'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:06.868435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1167/2337] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:15:06 2022] DELETE /identity/v3/users/80de40d2a02343858849d481590aa570 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:07.065369 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf02d8b-2de3-44d9-a135-aa4544c87efd service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:07.076523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf02d8b-2de3-44d9-a135-aa4544c87efd service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:07.120581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:07.122122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:07.164447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:07.165466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:07.165812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:07.166148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] PATH_INFO: `/v3/users/a84974cd49eb4902a81df688722acb78` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:07.167467 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a84974cd49eb4902a81df688722acb78)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:07.169642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:07.226385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbec52-899a-4791-942b-bff4b97cb2b6 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:07.231454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:07.232575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:07.232695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:07.235366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] PATH_INFO: `/v3/projects/71b68c04578747d08842aab556e98030` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:07.235366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b68c04578747d08842aab556e98030)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:07.236357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:07.339479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:07.353155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a84974cd49eb4902a81df688722acb78'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.356234 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b68c04578747d08842aab556e98030', 'request_id': 'req-321aead8-a7e4-4c6a-a9c2-368fb134cb75'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.360647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a84974cd49eb4902a81df688722acb78'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.361382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a84974cd49eb4902a81df688722acb78'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.361977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a84974cd49eb4902a81df688722acb78'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.362836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b68c04578747d08842aab556e98030', 'request_id': 'req-321aead8-a7e4-4c6a-a9c2-368fb134cb75'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.374832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-1a57ce62-a8de-4cfd-9e23-928f992bbe39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a84974cd49eb4902a81df688722acb78'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.439820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b68c04578747d08842aab556e98030', 'request_id': 'req-321aead8-a7e4-4c6a-a9c2-368fb134cb75'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.451374 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] The token cache is being invalidate because project 71b68c04578747d08842aab556e98030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:15:07.451660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.452280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.453873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1168/2338] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:15:07 2022] DELETE /identity/v3/projects/71b68c04578747d08842aab556e98030 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:15:07.457638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:07.458919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:07.522448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aead8-a7e4-4c6a-a9c2-368fb134cb75 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:07.527097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:07.529305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:07.529305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:07.529305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] PATH_INFO: `/v3/users/13de76cf9f5742e0bda36a6dee992410` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:07.529685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13de76cf9f5742e0bda36a6dee992410)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:07.531640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:07.554383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:07.557996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1171/2339] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:15:07 2022] DELETE /identity/v3/users/a84974cd49eb4902a81df688722acb78 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:07.559781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425259a-a49a-4a7d-ab82-528f7b75ec61 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:07.560000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425259a-a49a-4a7d-ab82-528f7b75ec61 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:07.560276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425259a-a49a-4a7d-ab82-528f7b75ec61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:07.578290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c425259a-a49a-4a7d-ab82-528f7b75ec61 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:07.579492 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c425259a-a49a-4a7d-ab82-528f7b75ec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:07.642214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1172/2340] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:15:07.651714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425259a-a49a-4a7d-ab82-528f7b75ec61 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:07.653663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c425259a-a49a-4a7d-ab82-528f7b75ec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:07.712607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425259a-a49a-4a7d-ab82-528f7b75ec61 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:07.716409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5ff95-412f-449a-90ee-d465e824347c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:07.717110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5ff95-412f-449a-90ee-d465e824347c service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:07.721114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5ff95-412f-449a-90ee-d465e824347c service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:07.721114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5ff95-412f-449a-90ee-d465e824347c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:07.728588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5ff95-412f-449a-90ee-d465e824347c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:07.730228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5ff95-412f-449a-90ee-d465e824347c service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:07.740702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1173/2341] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:15:07.745845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13de76cf9f5742e0bda36a6dee992410'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.753406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13de76cf9f5742e0bda36a6dee992410'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.754161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13de76cf9f5742e0bda36a6dee992410'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.767808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13de76cf9f5742e0bda36a6dee992410'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.768691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13de76cf9f5742e0bda36a6dee992410'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:07.876959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1169/2342] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:15:07 2022] DELETE /identity/v3/users/13de76cf9f5742e0bda36a6dee992410 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:15:08.179187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5ff95-412f-449a-90ee-d465e824347c service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:08.190571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5ff95-412f-449a-90ee-d465e824347c service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:08.238205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:08.240335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:08.248150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:08.258730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:08.259600 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:08.259883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:08.260154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] PATH_INFO: `/v3/projects/c763e54bb34246659fca207127eb7a9f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:08.261446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c763e54bb34246659fca207127eb7a9f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:08.263774 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:08.323114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1eb65b-bd8b-4f34-8b0b-e5c88811de14 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:08.327673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:08.328520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:08.328827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:08.329101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] PATH_INFO: `/v3/projects/fed87ad05186438181094010f1abd7e7` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:08.330253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fed87ad05186438181094010f1abd7e7)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:08.332276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:08.404790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c763e54bb34246659fca207127eb7a9f', 'request_id': 'req-88732576-7000-4662-985e-bb14f31bb18a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.410229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c763e54bb34246659fca207127eb7a9f', 'request_id': 'req-88732576-7000-4662-985e-bb14f31bb18a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.416450 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c763e54bb34246659fca207127eb7a9f', 'request_id': 'req-88732576-7000-4662-985e-bb14f31bb18a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.446966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fed87ad05186438181094010f1abd7e7', 'request_id': 'req-5bb1f801-56f2-4701-b459-992b556c17dd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.453015 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fed87ad05186438181094010f1abd7e7', 'request_id': 'req-5bb1f801-56f2-4701-b459-992b556c17dd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.502811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] The token cache is being invalidate because project c763e54bb34246659fca207127eb7a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:15:08.503100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.503707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.505319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1174/2343] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:15:08 2022] DELETE /identity/v3/projects/c763e54bb34246659fca207127eb7a9f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:15:08.549377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fed87ad05186438181094010f1abd7e7', 'request_id': 'req-5bb1f801-56f2-4701-b459-992b556c17dd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.564099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] The token cache is being invalidate because project fed87ad05186438181094010f1abd7e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:15:08.564487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.565350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.568004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1170/2344] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:15:08 2022] DELETE /identity/v3/projects/fed87ad05186438181094010f1abd7e7 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:08.571486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:08.573310 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:08.644249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732576-7000-4662-985e-bb14f31bb18a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:08.707453 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:08.708823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:08.709790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:08.710075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] PATH_INFO: `/v3/users/54142fcb4444418398bc1bcfe2aeaa09` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:08.711362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54142fcb4444418398bc1bcfe2aeaa09)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:08.713362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:08.904827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:08.917854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54142fcb4444418398bc1bcfe2aeaa09'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.924815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54142fcb4444418398bc1bcfe2aeaa09'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.925484 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54142fcb4444418398bc1bcfe2aeaa09'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.926121 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54142fcb4444418398bc1bcfe2aeaa09'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:08.937912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2b690d81-bf8c-4e1a-ae52-48739e5ec4f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54142fcb4444418398bc1bcfe2aeaa09'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:09.037677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1175/2345] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:15:08 2022] DELETE /identity/v3/users/54142fcb4444418398bc1bcfe2aeaa09 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:10.227182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:10.227182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:10.242565 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1f801-56f2-4701-b459-992b556c17dd admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:10.276631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:10.314541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:10.315448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:10.315734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:10.316012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] PATH_INFO: `/v3/projects/eff1faee2cc64482a8fa3b54f391c0b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:10.317232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff1faee2cc64482a8fa3b54f391c0b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:10.319720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:10.458579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff1faee2cc64482a8fa3b54f391c0b3', 'request_id': 'req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:10.464789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff1faee2cc64482a8fa3b54f391c0b3', 'request_id': 'req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:10.577304 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff1faee2cc64482a8fa3b54f391c0b3', 'request_id': 'req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:10.588530 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] The token cache is being invalidate because project eff1faee2cc64482a8fa3b54f391c0b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:15:10.592923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:10.592923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b49c4f4e-6834-4c4e-8262-6e0dcef03fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:10.592923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1171/2346] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:15:10 2022] DELETE /identity/v3/projects/eff1faee2cc64482a8fa3b54f391c0b3 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:10.596099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b73e81-7675-4de9-925c-c04ec868a783 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:10.596625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b73e81-7675-4de9-925c-c04ec868a783 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:10.596730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b73e81-7675-4de9-925c-c04ec868a783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:10.617107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-00b73e81-7675-4de9-925c-c04ec868a783 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:10.618294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-00b73e81-7675-4de9-925c-c04ec868a783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:10.675111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1176/2347] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:10.683933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820943f-e727-4c2e-bd5b-db888dd48e8d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:10.684238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820943f-e727-4c2e-bd5b-db888dd48e8d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:10.684522 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820943f-e727-4c2e-bd5b-db888dd48e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:10.703029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1820943f-e727-4c2e-bd5b-db888dd48e8d None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:10.704294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1820943f-e727-4c2e-bd5b-db888dd48e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:10.712862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1172/2348] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:15:10.717047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b73e81-7675-4de9-925c-c04ec868a783 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:10.727284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b73e81-7675-4de9-925c-c04ec868a783 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:10.731856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ae3b-53f7-456c-93d2-482006ebada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:10.732483 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ae3b-53f7-456c-93d2-482006ebada3 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:10.732885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ae3b-53f7-456c-93d2-482006ebada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:10.733070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ae3b-53f7-456c-93d2-482006ebada3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:10.733794 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257ae3b-53f7-456c-93d2-482006ebada3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:15:10.734100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257ae3b-53f7-456c-93d2-482006ebada3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:10.736106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257ae3b-53f7-456c-93d2-482006ebada3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:10.748663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1177/2349] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:15:10.752864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820943f-e727-4c2e-bd5b-db888dd48e8d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:10.762051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820943f-e727-4c2e-bd5b-db888dd48e8d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:10.766729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e73ef8-535f-4bbc-9a03-2c32e3de8bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:10.769781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e73ef8-535f-4bbc-9a03-2c32e3de8bca admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:10.769781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e73ef8-535f-4bbc-9a03-2c32e3de8bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:10.769781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e73ef8-535f-4bbc-9a03-2c32e3de8bca admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:10.769781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e73ef8-535f-4bbc-9a03-2c32e3de8bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:10.770605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e73ef8-535f-4bbc-9a03-2c32e3de8bca admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:10.876288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1173/2350] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:15:10 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:10.880283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ae3b-53f7-456c-93d2-482006ebada3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:10.890376 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ae3b-53f7-456c-93d2-482006ebada3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:10.946962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d26a62-d6f3-47a2-8f8e-1a07065804fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:10.947591 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d26a62-d6f3-47a2-8f8e-1a07065804fd admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:10.947876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d26a62-d6f3-47a2-8f8e-1a07065804fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:10.948154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d26a62-d6f3-47a2-8f8e-1a07065804fd admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:10.948855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d26a62-d6f3-47a2-8f8e-1a07065804fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:10.950749 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d26a62-d6f3-47a2-8f8e-1a07065804fd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:11.034100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1178/2351] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:15:10 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:11.038836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e73ef8-535f-4bbc-9a03-2c32e3de8bca admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:11.049151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e73ef8-535f-4bbc-9a03-2c32e3de8bca admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:11.053846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935cd44-c1b2-4411-ad41-94e093409ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:11.054500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935cd44-c1b2-4411-ad41-94e093409ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.054781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935cd44-c1b2-4411-ad41-94e093409ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.055054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935cd44-c1b2-4411-ad41-94e093409ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.055838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935cd44-c1b2-4411-ad41-94e093409ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:11.057462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935cd44-c1b2-4411-ad41-94e093409ae1 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:11.067708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1174/2352] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:11.071567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d26a62-d6f3-47a2-8f8e-1a07065804fd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:11.081675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d26a62-d6f3-47a2-8f8e-1a07065804fd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:11.087285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b53d2-d48e-4071-8a53-e1c45b0c4ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:11.087935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b53d2-d48e-4071-8a53-e1c45b0c4ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.088226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b53d2-d48e-4071-8a53-e1c45b0c4ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.088507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b53d2-d48e-4071-8a53-e1c45b0c4ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.089283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b53d2-d48e-4071-8a53-e1c45b0c4ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:11.090990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b53d2-d48e-4071-8a53-e1c45b0c4ca9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:11.101689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1179/2353] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:11.107969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935cd44-c1b2-4411-ad41-94e093409ae1 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:11.117443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935cd44-c1b2-4411-ad41-94e093409ae1 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:11.123204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c38ea-7881-429f-bc11-ed6a0c2de4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:11.124713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c38ea-7881-429f-bc11-ed6a0c2de4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.125111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c38ea-7881-429f-bc11-ed6a0c2de4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.125545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c38ea-7881-429f-bc11-ed6a0c2de4b4 admin admin] PATH_INFO: `/v3/projects/44f3c891c8024ca98fbcdfe6478e14ce/users/d3b885ea5d444df695c2774cd5f6c57b/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.142996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c38ea-7881-429f-bc11-ed6a0c2de4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f3c891c8024ca98fbcdfe6478e14ce, user_id=d3b885ea5d444df695c2774cd5f6c57b, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:11.146337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c38ea-7881-429f-bc11-ed6a0c2de4b4 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:11.230117 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1175/2354] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:11 2022] PUT /identity/v3/projects/44f3c891c8024ca98fbcdfe6478e14ce/users/d3b885ea5d444df695c2774cd5f6c57b/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:11.240867 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83b3cc-1eed-4566-a6d3-b82937407bad None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.241250 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83b3cc-1eed-4566-a6d3-b82937407bad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.241624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83b3cc-1eed-4566-a6d3-b82937407bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.269140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-6a83b3cc-1eed-4566-a6d3-b82937407bad None None] MFA Rules not processed for user `d3b885ea5d444df695c2774cd5f6c57b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:11.271109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-6a83b3cc-1eed-4566-a6d3-b82937407bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:11.275172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:11.337033 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1180/2355] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:11.341738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c38ea-7881-429f-bc11-ed6a0c2de4b4 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:11.391875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:11.407213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c38ea-7881-429f-bc11-ed6a0c2de4b4 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:11.411795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3a994-0a33-4d0c-b2b9-9dc6e57b0b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:11.412444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3a994-0a33-4d0c-b2b9-9dc6e57b0b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.412726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3a994-0a33-4d0c-b2b9-9dc6e57b0b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.413029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3a994-0a33-4d0c-b2b9-9dc6e57b0b73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.413832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3a994-0a33-4d0c-b2b9-9dc6e57b0b73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:11.415755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3a994-0a33-4d0c-b2b9-9dc6e57b0b73 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:11.523404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1176/2356] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:15:11 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:11.527476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83b3cc-1eed-4566-a6d3-b82937407bad None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:11.538060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83b3cc-1eed-4566-a6d3-b82937407bad None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:11.595229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82558d9-17e8-411b-b078-d4161f517329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:11.595895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82558d9-17e8-411b-b078-d4161f517329 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.596182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82558d9-17e8-411b-b078-d4161f517329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.596757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82558d9-17e8-411b-b078-d4161f517329 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.597179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82558d9-17e8-411b-b078-d4161f517329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:11.599126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82558d9-17e8-411b-b078-d4161f517329 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:11.674628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1181/2357] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:15:11 2022] POST /identity/v3/users => generated 299 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:15:11.678779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3a994-0a33-4d0c-b2b9-9dc6e57b0b73 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:11.689157 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3a994-0a33-4d0c-b2b9-9dc6e57b0b73 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:11.694250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e0f1e-2738-4aff-9b8d-55da5e0c42e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:11.694896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e0f1e-2738-4aff-9b8d-55da5e0c42e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.695179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e0f1e-2738-4aff-9b8d-55da5e0c42e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.695453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e0f1e-2738-4aff-9b8d-55da5e0c42e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.696168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e0f1e-2738-4aff-9b8d-55da5e0c42e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:11.697771 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e0f1e-2738-4aff-9b8d-55da5e0c42e2 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:11.708842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1177/2358] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:11.712842 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82558d9-17e8-411b-b078-d4161f517329 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:11.722677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82558d9-17e8-411b-b078-d4161f517329 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:11.727300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37104c9-795a-4b6e-82db-0877bc384ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:11.728198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37104c9-795a-4b6e-82db-0877bc384ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.728478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37104c9-795a-4b6e-82db-0877bc384ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.728750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37104c9-795a-4b6e-82db-0877bc384ecc admin admin] PATH_INFO: `/v3/projects/7610d3b63cc9484eae9c652c48b876ff/users/50b67d5912ac46c6ac986c7ce2e0712c/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.744311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37104c9-795a-4b6e-82db-0877bc384ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7610d3b63cc9484eae9c652c48b876ff, user_id=50b67d5912ac46c6ac986c7ce2e0712c, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:11.746983 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37104c9-795a-4b6e-82db-0877bc384ecc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:11.793865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1182/2359] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:11 2022] PUT /identity/v3/projects/7610d3b63cc9484eae9c652c48b876ff/users/50b67d5912ac46c6ac986c7ce2e0712c/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:11.797715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e0f1e-2738-4aff-9b8d-55da5e0c42e2 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:11.807694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e0f1e-2738-4aff-9b8d-55da5e0c42e2 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:11.863518 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dc74d-5e14-4a41-9927-399d17722f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:11.864227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dc74d-5e14-4a41-9927-399d17722f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.864528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dc74d-5e14-4a41-9927-399d17722f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.864823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dc74d-5e14-4a41-9927-399d17722f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.865534 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dc74d-5e14-4a41-9927-399d17722f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:11.867121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dc74d-5e14-4a41-9927-399d17722f27 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:11.876538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1178/2360] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:11.880708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37104c9-795a-4b6e-82db-0877bc384ecc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:11.890985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37104c9-795a-4b6e-82db-0877bc384ecc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:11.896161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a014c6-9bed-4779-b725-8637822b93c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:11.897063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a014c6-9bed-4779-b725-8637822b93c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.897345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a014c6-9bed-4779-b725-8637822b93c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.897622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a014c6-9bed-4779-b725-8637822b93c0 admin admin] PATH_INFO: `/v3/projects/7610d3b63cc9484eae9c652c48b876ff/users/50b67d5912ac46c6ac986c7ce2e0712c/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.899592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a014c6-9bed-4779-b725-8637822b93c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7610d3b63cc9484eae9c652c48b876ff, user_id=50b67d5912ac46c6ac986c7ce2e0712c, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:11.902261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a014c6-9bed-4779-b725-8637822b93c0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:11.913020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1183/2361] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:11 2022] PUT /identity/v3/projects/7610d3b63cc9484eae9c652c48b876ff/users/50b67d5912ac46c6ac986c7ce2e0712c/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:11.923870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd6cd0-a72a-4dfd-b61c-3bd3fa0468b9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:11.924161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd6cd0-a72a-4dfd-b61c-3bd3fa0468b9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:11.924439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd6cd0-a72a-4dfd-b61c-3bd3fa0468b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:11.950123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-08dd6cd0-a72a-4dfd-b61c-3bd3fa0468b9 None None] MFA Rules not processed for user `50b67d5912ac46c6ac986c7ce2e0712c`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:11.951318 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-08dd6cd0-a72a-4dfd-b61c-3bd3fa0468b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:12.018986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1179/2362] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:12.031659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a014c6-9bed-4779-b725-8637822b93c0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:12.039076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-25a014c6-9bed-4779-b725-8637822b93c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:12.061571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e05bfa-68f5-4bd5-bb64-f23b590efb3b None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:12.061861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e05bfa-68f5-4bd5-bb64-f23b590efb3b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:12.062144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e05bfa-68f5-4bd5-bb64-f23b590efb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:12.080376 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-21e05bfa-68f5-4bd5-bb64-f23b590efb3b None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:12.081637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-21e05bfa-68f5-4bd5-bb64-f23b590efb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:12.100741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a014c6-9bed-4779-b725-8637822b93c0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:12.104426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72988755-682f-4941-977a-693668876f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:12.105115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72988755-682f-4941-977a-693668876f23 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:12.105404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72988755-682f-4941-977a-693668876f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:12.105688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72988755-682f-4941-977a-693668876f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:12.115253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72988755-682f-4941-977a-693668876f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:12.116699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72988755-682f-4941-977a-693668876f23 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:12.119920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:12.122483 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:12.127502 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1184/2363] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:15:12.137638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1180/2364] 10.222.0.45 () {58 vars in 967 bytes} [Tue Jul 5 16:15:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:15:12.150513 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72988755-682f-4941-977a-693668876f23 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:12.153404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-72988755-682f-4941-977a-693668876f23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:12.195115 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e05bfa-68f5-4bd5-bb64-f23b590efb3b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:12.229161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72988755-682f-4941-977a-693668876f23 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:12.234315 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d8252-7304-4cb4-9cf8-f59006c3961e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:12.236524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8252-7304-4cb4-9cf8-f59006c3961e service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:12.236524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8252-7304-4cb4-9cf8-f59006c3961e service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:12.236524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8252-7304-4cb4-9cf8-f59006c3961e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:12.244277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d8252-7304-4cb4-9cf8-f59006c3961e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:12.245909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d8252-7304-4cb4-9cf8-f59006c3961e service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:12.254216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1185/2365] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:15:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:15:12.257436 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e05bfa-68f5-4bd5-bb64-f23b590efb3b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:12.262720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:12.264794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:12.264794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:12.264794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] PATH_INFO: `/v3/users/d3b885ea5d444df695c2774cd5f6c57b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:12.266609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b885ea5d444df695c2774cd5f6c57b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:12.268932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:12.481079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b885ea5d444df695c2774cd5f6c57b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:12.488449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b885ea5d444df695c2774cd5f6c57b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:12.489720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b885ea5d444df695c2774cd5f6c57b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:12.503709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b885ea5d444df695c2774cd5f6c57b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:12.504716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b885ea5d444df695c2774cd5f6c57b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:12.575455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1181/2366] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:15:12 2022] DELETE /identity/v3/users/d3b885ea5d444df695c2774cd5f6c57b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:15:12.581858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d47f29-4992-40df-a42d-f9756714d3c3 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:12.581858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d47f29-4992-40df-a42d-f9756714d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:12.581858 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d47f29-4992-40df-a42d-f9756714d3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:12.599481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c9d47f29-4992-40df-a42d-f9756714d3c3 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:12.600654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c9d47f29-4992-40df-a42d-f9756714d3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:12.658237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1186/2367] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:12.668570 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:12.669958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:12.729153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474e318-6f9a-44e5-aed3-c43ab8305cf4 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:12.732806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73e883-942c-4290-ac99-a60989907578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:12.733452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73e883-942c-4290-ac99-a60989907578 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:12.733734 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73e883-942c-4290-ac99-a60989907578 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:12.734008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73e883-942c-4290-ac99-a60989907578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:12.743258 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73e883-942c-4290-ac99-a60989907578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:12.744682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73e883-942c-4290-ac99-a60989907578 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:12.754108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1182/2368] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:15:13.190944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d47f29-4992-40df-a42d-f9756714d3c3 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:13.190944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c9d47f29-4992-40df-a42d-f9756714d3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:13.197332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d47f29-4992-40df-a42d-f9756714d3c3 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:13.203115 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:13.204228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:13.204633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:13.205069 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] PATH_INFO: `/v3/projects/44f3c891c8024ca98fbcdfe6478e14ce` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:13.206414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f3c891c8024ca98fbcdfe6478e14ce)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:13.216389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:13.326143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f3c891c8024ca98fbcdfe6478e14ce', 'request_id': 'req-bb00858d-7eb2-481c-9b29-6526d8ade7df'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:13.331591 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f3c891c8024ca98fbcdfe6478e14ce', 'request_id': 'req-bb00858d-7eb2-481c-9b29-6526d8ade7df'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:13.338132 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f3c891c8024ca98fbcdfe6478e14ce', 'request_id': 'req-bb00858d-7eb2-481c-9b29-6526d8ade7df'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:13.399813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] The token cache is being invalidate because project 44f3c891c8024ca98fbcdfe6478e14ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:15:13.400099 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:13.400732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:13.402330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1187/2369] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:15:13 2022] DELETE /identity/v3/projects/44f3c891c8024ca98fbcdfe6478e14ce => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:13.406338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73e883-942c-4290-ac99-a60989907578 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:13.416055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73e883-942c-4290-ac99-a60989907578 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:13.420593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:13.473981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:13.474907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:13.475198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:13.475520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] PATH_INFO: `/v3/users/50b67d5912ac46c6ac986c7ce2e0712c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:13.477044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b67d5912ac46c6ac986c7ce2e0712c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:13.479909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:13.655198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b67d5912ac46c6ac986c7ce2e0712c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:13.663683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b67d5912ac46c6ac986c7ce2e0712c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:13.664366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b67d5912ac46c6ac986c7ce2e0712c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:13.676221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b67d5912ac46c6ac986c7ce2e0712c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:13.676927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-44569752-8342-4479-8a5b-32c7718e388b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b67d5912ac46c6ac986c7ce2e0712c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:13.766993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1183/2370] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:15:13 2022] DELETE /identity/v3/users/50b67d5912ac46c6ac986c7ce2e0712c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:14.201404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:14.202924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:14.217938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00858d-7eb2-481c-9b29-6526d8ade7df admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:14.231564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:14.277194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:14.278000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:14.278281 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:14.278554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] PATH_INFO: `/v3/projects/7610d3b63cc9484eae9c652c48b876ff` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:14.279729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7610d3b63cc9484eae9c652c48b876ff)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:14.281695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:14.376065 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7610d3b63cc9484eae9c652c48b876ff', 'request_id': 'req-4b48cd8b-3c28-4438-87e4-e59ba6584294'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:14.382167 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7610d3b63cc9484eae9c652c48b876ff', 'request_id': 'req-4b48cd8b-3c28-4438-87e4-e59ba6584294'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:14.388368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7610d3b63cc9484eae9c652c48b876ff', 'request_id': 'req-4b48cd8b-3c28-4438-87e4-e59ba6584294'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:14.466710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] The token cache is being invalidate because project 7610d3b63cc9484eae9c652c48b876ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:15:14.466996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:14.467607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-4b48cd8b-3c28-4438-87e4-e59ba6584294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:14.469173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1188/2371] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:15:14 2022] DELETE /identity/v3/projects/7610d3b63cc9484eae9c652c48b876ff => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:14.474354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caefe9-0d1c-41b1-8e65-f4c839767cb6 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:14.474841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caefe9-0d1c-41b1-8e65-f4c839767cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:14.474936 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caefe9-0d1c-41b1-8e65-f4c839767cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:14.494340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-72caefe9-0d1c-41b1-8e65-f4c839767cb6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:14.495527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-72caefe9-0d1c-41b1-8e65-f4c839767cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:14.552818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1184/2372] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:14.560495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b095b6d-486e-46d8-80c5-38563b453f45 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:14.560809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b095b6d-486e-46d8-80c5-38563b453f45 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:14.561098 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b095b6d-486e-46d8-80c5-38563b453f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:14.582318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3b095b6d-486e-46d8-80c5-38563b453f45 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:14.584883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3b095b6d-486e-46d8-80c5-38563b453f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:14.592404 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1189/2373] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:14.596717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72caefe9-0d1c-41b1-8e65-f4c839767cb6 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:14.606677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72caefe9-0d1c-41b1-8e65-f4c839767cb6 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:14.612113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe0796-7a01-4b03-a319-8b9e09453e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:14.612806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe0796-7a01-4b03-a319-8b9e09453e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:14.613093 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe0796-7a01-4b03-a319-8b9e09453e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:14.613367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe0796-7a01-4b03-a319-8b9e09453e65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:14.614213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe0796-7a01-4b03-a319-8b9e09453e65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:15:14.614667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe0796-7a01-4b03-a319-8b9e09453e65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:14.617152 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe0796-7a01-4b03-a319-8b9e09453e65 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:14.630999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1185/2374] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:15:14.635964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b095b6d-486e-46d8-80c5-38563b453f45 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:14.652863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b095b6d-486e-46d8-80c5-38563b453f45 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:14.657827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793050d-c2b3-41a9-acae-43edcd1b38e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:14.659141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793050d-c2b3-41a9-acae-43edcd1b38e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:14.659422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793050d-c2b3-41a9-acae-43edcd1b38e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:14.659698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793050d-c2b3-41a9-acae-43edcd1b38e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:14.660475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793050d-c2b3-41a9-acae-43edcd1b38e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:14.662868 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793050d-c2b3-41a9-acae-43edcd1b38e8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:14.779879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1190/2375] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:15:14 2022] POST /identity/v3/projects => generated 399 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:15:14.786408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe0796-7a01-4b03-a319-8b9e09453e65 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:14.798311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe0796-7a01-4b03-a319-8b9e09453e65 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:14.814178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:14.874353 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea0c58-1f86-4637-9a55-5cda6de5b96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:14.875079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea0c58-1f86-4637-9a55-5cda6de5b96b admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:14.875496 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea0c58-1f86-4637-9a55-5cda6de5b96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:14.875795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea0c58-1f86-4637-9a55-5cda6de5b96b admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:14.876474 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea0c58-1f86-4637-9a55-5cda6de5b96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:14.878832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea0c58-1f86-4637-9a55-5cda6de5b96b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:14.983676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1186/2376] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:15:14 2022] POST /identity/v3/users => generated 300 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:14.988123 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793050d-c2b3-41a9-acae-43edcd1b38e8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:14.998832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793050d-c2b3-41a9-acae-43edcd1b38e8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:15.003355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818df4cf-51ea-4c76-b1f4-1e89377e005f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:15.004013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818df4cf-51ea-4c76-b1f4-1e89377e005f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:15.004298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818df4cf-51ea-4c76-b1f4-1e89377e005f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:15.004597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818df4cf-51ea-4c76-b1f4-1e89377e005f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:15.005912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818df4cf-51ea-4c76-b1f4-1e89377e005f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:15.008301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818df4cf-51ea-4c76-b1f4-1e89377e005f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:15.018873 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1191/2377] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:15.022998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea0c58-1f86-4637-9a55-5cda6de5b96b admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:15.033436 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea0c58-1f86-4637-9a55-5cda6de5b96b admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:15.038128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f6b35-e648-4589-9cdc-24a34da29897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:15.041033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f6b35-e648-4589-9cdc-24a34da29897 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:15.041033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f6b35-e648-4589-9cdc-24a34da29897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:15.041033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f6b35-e648-4589-9cdc-24a34da29897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:15.041033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f6b35-e648-4589-9cdc-24a34da29897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:15.041648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f6b35-e648-4589-9cdc-24a34da29897 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:15.052593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1187/2378] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:15.057670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818df4cf-51ea-4c76-b1f4-1e89377e005f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:15.068629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818df4cf-51ea-4c76-b1f4-1e89377e005f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:15.073879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b741d-0b63-45eb-aeef-83183b7325be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:15.074775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b741d-0b63-45eb-aeef-83183b7325be admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:15.075145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b741d-0b63-45eb-aeef-83183b7325be admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:15.075426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b741d-0b63-45eb-aeef-83183b7325be admin admin] PATH_INFO: `/v3/projects/dd51c3cf132343b5b27874713068fce9/users/97ed0f36d0e94303bcd8d913d9ee0276/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:15.092218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b741d-0b63-45eb-aeef-83183b7325be admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd51c3cf132343b5b27874713068fce9, user_id=97ed0f36d0e94303bcd8d913d9ee0276, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:15.094926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b741d-0b63-45eb-aeef-83183b7325be admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:15.184309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1192/2379] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:15 2022] PUT /identity/v3/projects/dd51c3cf132343b5b27874713068fce9/users/97ed0f36d0e94303bcd8d913d9ee0276/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:15.194737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df73244-96bc-40bd-bd9a-99a754eed720 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:15.195193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df73244-96bc-40bd-bd9a-99a754eed720 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:15.195470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df73244-96bc-40bd-bd9a-99a754eed720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:15.231050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1df73244-96bc-40bd-bd9a-99a754eed720 None None] MFA Rules not processed for user `97ed0f36d0e94303bcd8d913d9ee0276`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:15.236505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1df73244-96bc-40bd-bd9a-99a754eed720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:15.310497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1188/2380] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:15:15.315184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b741d-0b63-45eb-aeef-83183b7325be admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:15.358623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:15.378762 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b741d-0b63-45eb-aeef-83183b7325be admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:15.383516 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58742c50-3e38-4e67-a4d0-347c5b26f1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:15.384171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58742c50-3e38-4e67-a4d0-347c5b26f1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:15.384459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58742c50-3e38-4e67-a4d0-347c5b26f1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:15.384739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58742c50-3e38-4e67-a4d0-347c5b26f1f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:15.385503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58742c50-3e38-4e67-a4d0-347c5b26f1f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:15.387419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58742c50-3e38-4e67-a4d0-347c5b26f1f3 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:15.491114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1193/2381] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:15:15 2022] POST /identity/v3/projects => generated 401 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:15:15.495420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df73244-96bc-40bd-bd9a-99a754eed720 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:15.510188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df73244-96bc-40bd-bd9a-99a754eed720 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:15.575491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9dd2c-db33-4e0b-b4aa-b2779d5d05aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:15.576167 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9dd2c-db33-4e0b-b4aa-b2779d5d05aa admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:15.576451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9dd2c-db33-4e0b-b4aa-b2779d5d05aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:15.576727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9dd2c-db33-4e0b-b4aa-b2779d5d05aa admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:15.577612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9dd2c-db33-4e0b-b4aa-b2779d5d05aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:15.579669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9dd2c-db33-4e0b-b4aa-b2779d5d05aa admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:15.714963 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1189/2382] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:15:15 2022] POST /identity/v3/users => generated 307 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:15.719007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58742c50-3e38-4e67-a4d0-347c5b26f1f3 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:15.729813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58742c50-3e38-4e67-a4d0-347c5b26f1f3 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:15.734302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739882f-1580-4a9b-94d9-9e52419f0731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:15.735254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739882f-1580-4a9b-94d9-9e52419f0731 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:15.735574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739882f-1580-4a9b-94d9-9e52419f0731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:15.735939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739882f-1580-4a9b-94d9-9e52419f0731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:15.736681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739882f-1580-4a9b-94d9-9e52419f0731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:15.738328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739882f-1580-4a9b-94d9-9e52419f0731 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:15.749462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1194/2383] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:15.755709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9dd2c-db33-4e0b-b4aa-b2779d5d05aa admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:15.771023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9dd2c-db33-4e0b-b4aa-b2779d5d05aa admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:15.775638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e0ba1-b6cb-44ef-9c91-a68718ad7346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:15.781057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e0ba1-b6cb-44ef-9c91-a68718ad7346 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:15.781057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e0ba1-b6cb-44ef-9c91-a68718ad7346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:15.781057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e0ba1-b6cb-44ef-9c91-a68718ad7346 admin admin] PATH_INFO: `/v3/projects/479541e79c14404ba8e10c22e7a5685c/users/22ae4bdc4d4c4fa1beefbb439f82cd7b/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:15.791577 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:15.795731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e0ba1-b6cb-44ef-9c91-a68718ad7346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=479541e79c14404ba8e10c22e7a5685c, user_id=22ae4bdc4d4c4fa1beefbb439f82cd7b, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:15.798454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e0ba1-b6cb-44ef-9c91-a68718ad7346 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:15.908694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1190/2384] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:15 2022] PUT /identity/v3/projects/479541e79c14404ba8e10c22e7a5685c/users/22ae4bdc4d4c4fa1beefbb439f82cd7b/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:15.912758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739882f-1580-4a9b-94d9-9e52419f0731 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:15.922755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739882f-1580-4a9b-94d9-9e52419f0731 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:15.979851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcbabd-af52-4ac7-a9b8-0b3b714972b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:15.980531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcbabd-af52-4ac7-a9b8-0b3b714972b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:15.980876 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcbabd-af52-4ac7-a9b8-0b3b714972b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:15.981149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcbabd-af52-4ac7-a9b8-0b3b714972b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:15.981873 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcbabd-af52-4ac7-a9b8-0b3b714972b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:15.983484 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcbabd-af52-4ac7-a9b8-0b3b714972b9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:15.993679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1195/2385] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:15.997783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e0ba1-b6cb-44ef-9c91-a68718ad7346 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:16.008533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e0ba1-b6cb-44ef-9c91-a68718ad7346 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:16.013352 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594bfc-fa79-4502-9078-dbd948b6bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:16.014217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594bfc-fa79-4502-9078-dbd948b6bbb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:16.014497 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594bfc-fa79-4502-9078-dbd948b6bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:16.014772 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594bfc-fa79-4502-9078-dbd948b6bbb2 admin admin] PATH_INFO: `/v3/projects/479541e79c14404ba8e10c22e7a5685c/users/22ae4bdc4d4c4fa1beefbb439f82cd7b/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:16.016713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74594bfc-fa79-4502-9078-dbd948b6bbb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=479541e79c14404ba8e10c22e7a5685c, user_id=22ae4bdc4d4c4fa1beefbb439f82cd7b, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:16.019362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74594bfc-fa79-4502-9078-dbd948b6bbb2 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:16.030071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1191/2386] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:15 2022] PUT /identity/v3/projects/479541e79c14404ba8e10c22e7a5685c/users/22ae4bdc4d4c4fa1beefbb439f82cd7b/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:16.041262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3c280-f007-46eb-b204-2ef39ba84e9c None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:16.041559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3c280-f007-46eb-b204-2ef39ba84e9c None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:16.041851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3c280-f007-46eb-b204-2ef39ba84e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:16.068291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cfd3c280-f007-46eb-b204-2ef39ba84e9c None None] MFA Rules not processed for user `22ae4bdc4d4c4fa1beefbb439f82cd7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:16.069606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cfd3c280-f007-46eb-b204-2ef39ba84e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:16.156064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1196/2387] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:16 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:16.167168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594bfc-fa79-4502-9078-dbd948b6bbb2 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:16.168968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-74594bfc-fa79-4502-9078-dbd948b6bbb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:16.225966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594bfc-fa79-4502-9078-dbd948b6bbb2 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:16.229705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116e8d7-df56-4909-8b0b-20d72de11732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:16.230358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116e8d7-df56-4909-8b0b-20d72de11732 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:16.230647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116e8d7-df56-4909-8b0b-20d72de11732 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:16.230942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116e8d7-df56-4909-8b0b-20d72de11732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:16.241175 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116e8d7-df56-4909-8b0b-20d72de11732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:16.242620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116e8d7-df56-4909-8b0b-20d72de11732 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:16.252526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1192/2388] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:16 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:15:16.495346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3c280-f007-46eb-b204-2ef39ba84e9c None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:16.506110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3c280-f007-46eb-b204-2ef39ba84e9c None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:16.509682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967edad-b7c4-48d9-b2e1-fb343460f161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:16.510354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967edad-b7c4-48d9-b2e1-fb343460f161 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:16.510639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967edad-b7c4-48d9-b2e1-fb343460f161 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:16.510945 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967edad-b7c4-48d9-b2e1-fb343460f161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:16.543681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:16.569730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967edad-b7c4-48d9-b2e1-fb343460f161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:16.571328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967edad-b7c4-48d9-b2e1-fb343460f161 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:16.580754 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1197/2389] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:15:21.476842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116e8d7-df56-4909-8b0b-20d72de11732 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:21.537699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116e8d7-df56-4909-8b0b-20d72de11732 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:21.542121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:21.542965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:21.543243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:21.543515 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] PATH_INFO: `/v3/users/97ed0f36d0e94303bcd8d913d9ee0276` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:21.544664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ed0f36d0e94303bcd8d913d9ee0276)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:21.546632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:21.730402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ed0f36d0e94303bcd8d913d9ee0276'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:21.737359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ed0f36d0e94303bcd8d913d9ee0276'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:21.738059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ed0f36d0e94303bcd8d913d9ee0276'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:21.750087 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ed0f36d0e94303bcd8d913d9ee0276'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:21.750704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ed0f36d0e94303bcd8d913d9ee0276'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:21.815399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:21.821009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1193/2390] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:15:21 2022] DELETE /identity/v3/users/97ed0f36d0e94303bcd8d913d9ee0276 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:21.825751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141beda-c26b-4e36-ad08-a98a7971fc59 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:21.826226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141beda-c26b-4e36-ad08-a98a7971fc59 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:21.826320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141beda-c26b-4e36-ad08-a98a7971fc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:21.843422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1141beda-c26b-4e36-ad08-a98a7971fc59 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:21.844618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1141beda-c26b-4e36-ad08-a98a7971fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:21.902773 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1198/2391] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:21.913013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:21.923651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:21.972329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b326f3e-40cc-4e13-a95a-a4798c6750ed admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:21.976448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478a17b-f158-4f96-aeb3-b32b0af0c7c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:21.977107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478a17b-f158-4f96-aeb3-b32b0af0c7c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:21.977390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478a17b-f158-4f96-aeb3-b32b0af0c7c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:21.977664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478a17b-f158-4f96-aeb3-b32b0af0c7c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:21.986674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478a17b-f158-4f96-aeb3-b32b0af0c7c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:21.988023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478a17b-f158-4f96-aeb3-b32b0af0c7c2 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:21.997880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1194/2392] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:15:22.402038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141beda-c26b-4e36-ad08-a98a7971fc59 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:22.403504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1141beda-c26b-4e36-ad08-a98a7971fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:22.417012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141beda-c26b-4e36-ad08-a98a7971fc59 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:22.422187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:22.423045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:22.423321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:22.423593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] PATH_INFO: `/v3/projects/dd51c3cf132343b5b27874713068fce9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:22.424791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd51c3cf132343b5b27874713068fce9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:22.426732 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:22.526879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd51c3cf132343b5b27874713068fce9', 'request_id': 'req-3a75aad3-1017-4320-b817-5706c1126c8e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:22.532675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd51c3cf132343b5b27874713068fce9', 'request_id': 'req-3a75aad3-1017-4320-b817-5706c1126c8e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:22.539644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd51c3cf132343b5b27874713068fce9', 'request_id': 'req-3a75aad3-1017-4320-b817-5706c1126c8e'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:22.610169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] The token cache is being invalidate because project dd51c3cf132343b5b27874713068fce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:15:22.610472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:22.611053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:22.612581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1199/2393] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:15:22 2022] DELETE /identity/v3/projects/dd51c3cf132343b5b27874713068fce9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:22.616582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478a17b-f158-4f96-aeb3-b32b0af0c7c2 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:22.626277 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478a17b-f158-4f96-aeb3-b32b0af0c7c2 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:22.681798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:22.682661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:22.682945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:22.683220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] PATH_INFO: `/v3/users/22ae4bdc4d4c4fa1beefbb439f82cd7b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:22.684476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=22ae4bdc4d4c4fa1beefbb439f82cd7b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:22.686423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:22.898269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22ae4bdc4d4c4fa1beefbb439f82cd7b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:22.905540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22ae4bdc4d4c4fa1beefbb439f82cd7b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:22.906199 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22ae4bdc4d4c4fa1beefbb439f82cd7b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:22.922479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22ae4bdc4d4c4fa1beefbb439f82cd7b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:22.922479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f49f4d58-b580-40ee-b068-6570ad736a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22ae4bdc4d4c4fa1beefbb439f82cd7b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:22.988624 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1195/2394] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:15:22 2022] DELETE /identity/v3/users/22ae4bdc4d4c4fa1beefbb439f82cd7b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:23.397383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:23.397383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:23.410359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75aad3-1017-4320-b817-5706c1126c8e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:23.475730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:23.482393 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:23.483255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:23.483540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:23.483935 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] PATH_INFO: `/v3/projects/479541e79c14404ba8e10c22e7a5685c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:23.485452 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=479541e79c14404ba8e10c22e7a5685c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:23.487410 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:23.627401 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '479541e79c14404ba8e10c22e7a5685c', 'request_id': 'req-12bca691-2326-4a5a-93c5-9414b52e21d6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:23.632673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '479541e79c14404ba8e10c22e7a5685c', 'request_id': 'req-12bca691-2326-4a5a-93c5-9414b52e21d6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:23.638722 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '479541e79c14404ba8e10c22e7a5685c', 'request_id': 'req-12bca691-2326-4a5a-93c5-9414b52e21d6'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:23.713720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] The token cache is being invalidate because project 479541e79c14404ba8e10c22e7a5685c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:15:23.714015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:23.714647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-12bca691-2326-4a5a-93c5-9414b52e21d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:23.716222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1200/2395] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:15:23 2022] DELETE /identity/v3/projects/479541e79c14404ba8e10c22e7a5685c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:23.722800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3ad5d-ccad-4e62-8955-6fad301e104e None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:23.723359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3ad5d-ccad-4e62-8955-6fad301e104e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:23.723359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3ad5d-ccad-4e62-8955-6fad301e104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:23.742325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-9ac3ad5d-ccad-4e62-8955-6fad301e104e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:23.743505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9ac3ad5d-ccad-4e62-8955-6fad301e104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:23.756179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:23.803708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1196/2396] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:15:23.811621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f882-1c87-4299-8cf0-146f5ac1bda7 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:23.811894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f882-1c87-4299-8cf0-146f5ac1bda7 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:23.812177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7f882-1c87-4299-8cf0-146f5ac1bda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:23.831943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-13c7f882-1c87-4299-8cf0-146f5ac1bda7 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:23.833690 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-13c7f882-1c87-4299-8cf0-146f5ac1bda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:23.841576 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1201/2397] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:15:23.845888 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3ad5d-ccad-4e62-8955-6fad301e104e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:23.864016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3ad5d-ccad-4e62-8955-6fad301e104e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:23.864016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d19d60-040e-472d-9964-b975b2e051ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:23.864016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d19d60-040e-472d-9964-b975b2e051ab admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:23.864016 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d19d60-040e-472d-9964-b975b2e051ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:23.864399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d19d60-040e-472d-9964-b975b2e051ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:23.864399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d19d60-040e-472d-9964-b975b2e051ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:15:23.864399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d19d60-040e-472d-9964-b975b2e051ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:23.865486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d19d60-040e-472d-9964-b975b2e051ab admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:23.882845 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1197/2398] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:15:23.884298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7f882-1c87-4299-8cf0-146f5ac1bda7 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:23.896245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7f882-1c87-4299-8cf0-146f5ac1bda7 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:23.900996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7282c10-4a40-46c9-a54f-2cf0b6690c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:23.901641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7282c10-4a40-46c9-a54f-2cf0b6690c20 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:23.901923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7282c10-4a40-46c9-a54f-2cf0b6690c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:23.902199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7282c10-4a40-46c9-a54f-2cf0b6690c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:23.902881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7282c10-4a40-46c9-a54f-2cf0b6690c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:23.904758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7282c10-4a40-46c9-a54f-2cf0b6690c20 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:24.001790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1202/2399] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:15:23 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:15:24.006149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d19d60-040e-472d-9964-b975b2e051ab admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:24.015899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d19d60-040e-472d-9964-b975b2e051ab admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:24.073475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4216ca-c4f7-4ff6-997b-c7b414329ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:24.074265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4216ca-c4f7-4ff6-997b-c7b414329ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:24.074585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4216ca-c4f7-4ff6-997b-c7b414329ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:24.074903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4216ca-c4f7-4ff6-997b-c7b414329ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:24.075600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4216ca-c4f7-4ff6-997b-c7b414329ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:24.077665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4216ca-c4f7-4ff6-997b-c7b414329ca8 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:24.154869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1198/2400] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:15:24 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:24.158910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7282c10-4a40-46c9-a54f-2cf0b6690c20 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:24.170190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7282c10-4a40-46c9-a54f-2cf0b6690c20 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:24.175007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85570dce-75f3-4a1e-bd39-5e2b80ba38ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:24.175663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85570dce-75f3-4a1e-bd39-5e2b80ba38ed admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:24.175950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85570dce-75f3-4a1e-bd39-5e2b80ba38ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:24.176226 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85570dce-75f3-4a1e-bd39-5e2b80ba38ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:24.177662 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85570dce-75f3-4a1e-bd39-5e2b80ba38ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:24.179265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85570dce-75f3-4a1e-bd39-5e2b80ba38ed admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:24.189819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1203/2401] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:15:24.193855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4216ca-c4f7-4ff6-997b-c7b414329ca8 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:24.204573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4216ca-c4f7-4ff6-997b-c7b414329ca8 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:24.209269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7408c7-d5e6-4594-bec7-9511772290f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:24.210132 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7408c7-d5e6-4594-bec7-9511772290f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:24.210419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7408c7-d5e6-4594-bec7-9511772290f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:24.210696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7408c7-d5e6-4594-bec7-9511772290f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:24.211449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7408c7-d5e6-4594-bec7-9511772290f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:24.213070 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7408c7-d5e6-4594-bec7-9511772290f2 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:24.223002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1199/2402] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:24.227474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85570dce-75f3-4a1e-bd39-5e2b80ba38ed admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:24.237616 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85570dce-75f3-4a1e-bd39-5e2b80ba38ed admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:24.242825 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5b507-e782-451e-81b9-f8f851f73e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:24.243693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5b507-e782-451e-81b9-f8f851f73e75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:24.243976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5b507-e782-451e-81b9-f8f851f73e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:24.244278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5b507-e782-451e-81b9-f8f851f73e75 admin admin] PATH_INFO: `/v3/projects/99b2139a4341427b8500597ba46dd921/users/278a671c198041e584a15149f81d610a/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:24.258864 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5b507-e782-451e-81b9-f8f851f73e75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b2139a4341427b8500597ba46dd921, user_id=278a671c198041e584a15149f81d610a, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:24.261537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5b507-e782-451e-81b9-f8f851f73e75 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:24.377090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1204/2403] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:24 2022] PUT /identity/v3/projects/99b2139a4341427b8500597ba46dd921/users/278a671c198041e584a15149f81d610a/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:24.387196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e829fd-ae32-495c-a92d-81096cf98ce1 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:24.387490 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e829fd-ae32-495c-a92d-81096cf98ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:24.387769 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e829fd-ae32-495c-a92d-81096cf98ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:24.414539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e3e829fd-ae32-495c-a92d-81096cf98ce1 None None] MFA Rules not processed for user `278a671c198041e584a15149f81d610a`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:24.415768 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e3e829fd-ae32-495c-a92d-81096cf98ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:24.481779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1200/2404] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:24.534024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5b507-e782-451e-81b9-f8f851f73e75 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:24.535746 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-49e5b507-e782-451e-81b9-f8f851f73e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:24.594295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5b507-e782-451e-81b9-f8f851f73e75 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:24.598263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf10d0-a63f-4063-a8cf-4c2bd1f07afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:24.599791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf10d0-a63f-4063-a8cf-4c2bd1f07afd service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:24.600076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf10d0-a63f-4063-a8cf-4c2bd1f07afd service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:24.600480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf10d0-a63f-4063-a8cf-4c2bd1f07afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:24.609267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf10d0-a63f-4063-a8cf-4c2bd1f07afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:24.611251 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf10d0-a63f-4063-a8cf-4c2bd1f07afd service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:24.621268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1205/2405] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:15:34.478753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:34.617008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:45.728326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e829fd-ae32-495c-a92d-81096cf98ce1 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:45.792880 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e829fd-ae32-495c-a92d-81096cf98ce1 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:45.797535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:45.798399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:45.798683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:45.798957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] PATH_INFO: `/v3/users/278a671c198041e584a15149f81d610a` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:45.800136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=278a671c198041e584a15149f81d610a)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:45.802767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:46.012828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '278a671c198041e584a15149f81d610a'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:46.020551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '278a671c198041e584a15149f81d610a'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:46.021427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '278a671c198041e584a15149f81d610a'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:46.034146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '278a671c198041e584a15149f81d610a'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:46.034792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '278a671c198041e584a15149f81d610a'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:46.130407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1201/2406] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:15:45 2022] DELETE /identity/v3/users/278a671c198041e584a15149f81d610a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:46.134830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c9fd1-31b1-4e98-a1b3-9e123fa534ad None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:46.135344 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c9fd1-31b1-4e98-a1b3-9e123fa534ad None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:46.135437 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c9fd1-31b1-4e98-a1b3-9e123fa534ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:46.153589 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a88c9fd1-31b1-4e98-a1b3-9e123fa534ad None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:46.158182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a88c9fd1-31b1-4e98-a1b3-9e123fa534ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:46.216039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1206/2407] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:46.226642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:46.228316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:46.293239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59252213-2c8e-4f4e-a18b-a75a419abaaf admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:46.296940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480bc57-c53c-4856-95ab-6c23828db463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:46.297592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480bc57-c53c-4856-95ab-6c23828db463 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:46.297885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480bc57-c53c-4856-95ab-6c23828db463 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:46.298165 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480bc57-c53c-4856-95ab-6c23828db463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:46.307540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480bc57-c53c-4856-95ab-6c23828db463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:46.308986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480bc57-c53c-4856-95ab-6c23828db463 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:46.318532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1202/2408] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:15:46.885955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c9fd1-31b1-4e98-a1b3-9e123fa534ad None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:46.887593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a88c9fd1-31b1-4e98-a1b3-9e123fa534ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:46.900259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c9fd1-31b1-4e98-a1b3-9e123fa534ad None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:46.904965 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25828943-3247-405b-a517-bd76db53351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:46.907585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25828943-3247-405b-a517-bd76db53351b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:46.907585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25828943-3247-405b-a517-bd76db53351b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:46.907585 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25828943-3247-405b-a517-bd76db53351b admin admin] PATH_INFO: `/v3/projects/99b2139a4341427b8500597ba46dd921` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:46.913663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25828943-3247-405b-a517-bd76db53351b admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b2139a4341427b8500597ba46dd921)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:46.915626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25828943-3247-405b-a517-bd76db53351b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:47.041072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25828943-3247-405b-a517-bd76db53351b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b2139a4341427b8500597ba46dd921', 'request_id': 'req-25828943-3247-405b-a517-bd76db53351b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:47.046818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25828943-3247-405b-a517-bd76db53351b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b2139a4341427b8500597ba46dd921', 'request_id': 'req-25828943-3247-405b-a517-bd76db53351b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:47.053672 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25828943-3247-405b-a517-bd76db53351b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b2139a4341427b8500597ba46dd921', 'request_id': 'req-25828943-3247-405b-a517-bd76db53351b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:47.162707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25828943-3247-405b-a517-bd76db53351b admin admin] The token cache is being invalidate because project 99b2139a4341427b8500597ba46dd921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:15:47.163346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25828943-3247-405b-a517-bd76db53351b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:47.163792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25828943-3247-405b-a517-bd76db53351b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:47.165423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1207/2409] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:15:46 2022] DELETE /identity/v3/projects/99b2139a4341427b8500597ba46dd921 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:47.170371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5627-6567-45a6-b21d-9eee4b6159c0 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:47.170798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5627-6567-45a6-b21d-9eee4b6159c0 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:47.170954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5627-6567-45a6-b21d-9eee4b6159c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:47.189130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e3ed5627-6567-45a6-b21d-9eee4b6159c0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:47.190309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e3ed5627-6567-45a6-b21d-9eee4b6159c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:47.208044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:47.261755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1203/2410] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:15:47.264508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc729d-2623-4665-8d85-dc0724ce9b6b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:47.264914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc729d-2623-4665-8d85-dc0724ce9b6b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:47.265201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc729d-2623-4665-8d85-dc0724ce9b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:47.283908 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cbdc729d-2623-4665-8d85-dc0724ce9b6b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:47.285149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cbdc729d-2623-4665-8d85-dc0724ce9b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:47.293925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1208/2411] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:47.298195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed5627-6567-45a6-b21d-9eee4b6159c0 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:47.309134 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed5627-6567-45a6-b21d-9eee4b6159c0 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:47.314679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b7a8a-f71c-48d1-a7d2-47f06aad0141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:47.315359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b7a8a-f71c-48d1-a7d2-47f06aad0141 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:47.315643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b7a8a-f71c-48d1-a7d2-47f06aad0141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:47.315918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b7a8a-f71c-48d1-a7d2-47f06aad0141 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:47.316641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b7a8a-f71c-48d1-a7d2-47f06aad0141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:15:47.316974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b7a8a-f71c-48d1-a7d2-47f06aad0141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:47.318586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b7a8a-f71c-48d1-a7d2-47f06aad0141 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:47.330202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1204/2412] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:15:47.334447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc729d-2623-4665-8d85-dc0724ce9b6b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:47.343446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc729d-2623-4665-8d85-dc0724ce9b6b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:47.348032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331621c-9fce-4a72-8df4-c0910fea6dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:47.349267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331621c-9fce-4a72-8df4-c0910fea6dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:47.349550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331621c-9fce-4a72-8df4-c0910fea6dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:47.349824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331621c-9fce-4a72-8df4-c0910fea6dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:47.350500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9331621c-9fce-4a72-8df4-c0910fea6dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:47.352377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9331621c-9fce-4a72-8df4-c0910fea6dd6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:47.412566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1209/2413] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:15:47 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:47.417742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b7a8a-f71c-48d1-a7d2-47f06aad0141 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:47.429438 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b7a8a-f71c-48d1-a7d2-47f06aad0141 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:47.491232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c7c81-fe7a-4b95-a789-04630ff2a3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:47.491903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c7c81-fe7a-4b95-a789-04630ff2a3ae admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:47.492191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c7c81-fe7a-4b95-a789-04630ff2a3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:47.492473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c7c81-fe7a-4b95-a789-04630ff2a3ae admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:47.493219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c7c81-fe7a-4b95-a789-04630ff2a3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:47.495224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c7c81-fe7a-4b95-a789-04630ff2a3ae admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:47.570193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1205/2414] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:15:47 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:47.574337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331621c-9fce-4a72-8df4-c0910fea6dd6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:47.584949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331621c-9fce-4a72-8df4-c0910fea6dd6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:47.589528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe378d5d-b5ab-40a3-902e-2fb7c5e13bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:47.590185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe378d5d-b5ab-40a3-902e-2fb7c5e13bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:47.590471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe378d5d-b5ab-40a3-902e-2fb7c5e13bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:47.590755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe378d5d-b5ab-40a3-902e-2fb7c5e13bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:47.591481 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe378d5d-b5ab-40a3-902e-2fb7c5e13bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:47.593120 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe378d5d-b5ab-40a3-902e-2fb7c5e13bc2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:47.604969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1210/2415] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:47.608753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c7c81-fe7a-4b95-a789-04630ff2a3ae admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:47.619583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c7c81-fe7a-4b95-a789-04630ff2a3ae admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:47.624102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae134e7-bd6f-4d95-a048-5cd0ad438e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:47.624808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae134e7-bd6f-4d95-a048-5cd0ad438e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:47.625090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae134e7-bd6f-4d95-a048-5cd0ad438e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:47.625363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae134e7-bd6f-4d95-a048-5cd0ad438e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:47.626079 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae134e7-bd6f-4d95-a048-5cd0ad438e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:47.627693 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae134e7-bd6f-4d95-a048-5cd0ad438e32 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:47.637456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1206/2416] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:47.645164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe378d5d-b5ab-40a3-902e-2fb7c5e13bc2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:47.651951 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe378d5d-b5ab-40a3-902e-2fb7c5e13bc2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:47.656738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767025b-b684-49fc-bb26-ab0ea0042ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:47.657633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767025b-b684-49fc-bb26-ab0ea0042ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:47.657916 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767025b-b684-49fc-bb26-ab0ea0042ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:47.658195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767025b-b684-49fc-bb26-ab0ea0042ef4 admin admin] PATH_INFO: `/v3/projects/de07093c553b4f0391c85e5ea04cf7f1/users/fe5390b9df9b445380119b995f4c457d/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:47.674162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767025b-b684-49fc-bb26-ab0ea0042ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de07093c553b4f0391c85e5ea04cf7f1, user_id=fe5390b9df9b445380119b995f4c457d, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:47.676168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767025b-b684-49fc-bb26-ab0ea0042ef4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:47.748232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1211/2417] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:47 2022] PUT /identity/v3/projects/de07093c553b4f0391c85e5ea04cf7f1/users/fe5390b9df9b445380119b995f4c457d/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:15:47.753416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8130039-1c22-44db-9e3f-3f21a6fe7208 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:47.753712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8130039-1c22-44db-9e3f-3f21a6fe7208 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:47.753990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8130039-1c22-44db-9e3f-3f21a6fe7208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:47.772604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f8130039-1c22-44db-9e3f-3f21a6fe7208 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:47.773832 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f8130039-1c22-44db-9e3f-3f21a6fe7208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:47.831446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1207/2418] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:15:47.841554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767025b-b684-49fc-bb26-ab0ea0042ef4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:47.843252 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8767025b-b684-49fc-bb26-ab0ea0042ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:47.865902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:47.901574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767025b-b684-49fc-bb26-ab0ea0042ef4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:47.905243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3ae10-a075-41d5-9930-e81861fd436b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:47.905891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3ae10-a075-41d5-9930-e81861fd436b service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:47.906176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3ae10-a075-41d5-9930-e81861fd436b service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:47.906455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3ae10-a075-41d5-9930-e81861fd436b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:47.916243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3ae10-a075-41d5-9930-e81861fd436b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:47.917769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3ae10-a075-41d5-9930-e81861fd436b service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:47.928786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1212/2419] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:15:48.638236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8130039-1c22-44db-9e3f-3f21a6fe7208 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:48.639637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f8130039-1c22-44db-9e3f-3f21a6fe7208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:48.651361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8130039-1c22-44db-9e3f-3f21a6fe7208 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:48.654808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8f149-dcd4-402b-951d-28df5fb82b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:48.655653 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8f149-dcd4-402b-951d-28df5fb82b3d service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:48.655934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8f149-dcd4-402b-951d-28df5fb82b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:48.656211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8f149-dcd4-402b-951d-28df5fb82b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:48.658103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-aca8f149-dcd4-402b-951d-28df5fb82b3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:48.670459 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8f149-dcd4-402b-951d-28df5fb82b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:48.671885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8f149-dcd4-402b-951d-28df5fb82b3d service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:48.682663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1208/2420] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:15:53.312892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3ae10-a075-41d5-9930-e81861fd436b service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:53.314945 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-91e3ae10-a075-41d5-9930-e81861fd436b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:53.335586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3ae10-a075-41d5-9930-e81861fd436b service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:53.343335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:53.345062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:53.345539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:53.345824 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] PATH_INFO: `/v3/users/0c8512e7c1704fac9cc4fcfe3bf1f18b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:53.347135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c8512e7c1704fac9cc4fcfe3bf1f18b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:53.349518 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:53.559322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c8512e7c1704fac9cc4fcfe3bf1f18b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:53.567249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c8512e7c1704fac9cc4fcfe3bf1f18b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:53.569327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c8512e7c1704fac9cc4fcfe3bf1f18b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:53.569939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c8512e7c1704fac9cc4fcfe3bf1f18b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:53.582705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-d0e82688-8e45-4994-86c6-b75e0c56595c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c8512e7c1704fac9cc4fcfe3bf1f18b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:53.670306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1213/2421] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:15:53 2022] DELETE /identity/v3/users/0c8512e7c1704fac9cc4fcfe3bf1f18b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:54.003937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8f149-dcd4-402b-951d-28df5fb82b3d service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:54.014374 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8f149-dcd4-402b-951d-28df5fb82b3d service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:54.027405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:54.098122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:54.099784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:54.100262 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:54.100800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] PATH_INFO: `/v3/projects/42e6ad67dbf542528726c545872ac10c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:54.105760 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e6ad67dbf542528726c545872ac10c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:54.108248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:54.267326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e6ad67dbf542528726c545872ac10c', 'request_id': 'req-eab08282-e6e6-42c9-974e-50b1aee6b2b3'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:54.274677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e6ad67dbf542528726c545872ac10c', 'request_id': 'req-eab08282-e6e6-42c9-974e-50b1aee6b2b3'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:54.342030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e6ad67dbf542528726c545872ac10c', 'request_id': 'req-eab08282-e6e6-42c9-974e-50b1aee6b2b3'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:54.360904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] The token cache is being invalidate because project 42e6ad67dbf542528726c545872ac10c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:15:54.361717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:54.361964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eab08282-e6e6-42c9-974e-50b1aee6b2b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:15:54.363691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1209/2422] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:15:54 2022] DELETE /identity/v3/projects/42e6ad67dbf542528726c545872ac10c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:54.369465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798be00b-ff29-4993-ab2e-6f63ea80559e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:54.369997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798be00b-ff29-4993-ab2e-6f63ea80559e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:54.370103 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798be00b-ff29-4993-ab2e-6f63ea80559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:54.389460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-798be00b-ff29-4993-ab2e-6f63ea80559e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:54.390706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-798be00b-ff29-4993-ab2e-6f63ea80559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:54.481681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1214/2423] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:54.490000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595d003-2329-43b7-beee-b032771ac073 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:54.490264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595d003-2329-43b7-beee-b032771ac073 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:54.490557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595d003-2329-43b7-beee-b032771ac073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:54.519610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-a595d003-2329-43b7-beee-b032771ac073 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:54.521588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a595d003-2329-43b7-beee-b032771ac073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:54.533279 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1210/2424] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:54.534501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798be00b-ff29-4993-ab2e-6f63ea80559e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:54.546804 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798be00b-ff29-4993-ab2e-6f63ea80559e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:54.551977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278ef0-7db2-4bae-9e8e-21bf1933b933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:54.552684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278ef0-7db2-4bae-9e8e-21bf1933b933 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:54.553071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278ef0-7db2-4bae-9e8e-21bf1933b933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:54.553335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278ef0-7db2-4bae-9e8e-21bf1933b933 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:54.555509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9278ef0-7db2-4bae-9e8e-21bf1933b933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:15:54.555509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9278ef0-7db2-4bae-9e8e-21bf1933b933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:54.556201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9278ef0-7db2-4bae-9e8e-21bf1933b933 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:54.569589 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1215/2425] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:15:54.574159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595d003-2329-43b7-beee-b032771ac073 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:54.585397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595d003-2329-43b7-beee-b032771ac073 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:54.592510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff454922-6297-43a9-8817-63d0985ce096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:54.593263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454922-6297-43a9-8817-63d0985ce096 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:54.593560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454922-6297-43a9-8817-63d0985ce096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:54.593836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454922-6297-43a9-8817-63d0985ce096 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:54.594538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff454922-6297-43a9-8817-63d0985ce096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:54.610420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff454922-6297-43a9-8817-63d0985ce096 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:54.721349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1211/2426] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:15:54 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:54.729075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278ef0-7db2-4bae-9e8e-21bf1933b933 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:54.741697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278ef0-7db2-4bae-9e8e-21bf1933b933 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:54.766436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:54.821212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a256d16-258f-4876-9205-6941ccbb01b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:54.821915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a256d16-258f-4876-9205-6941ccbb01b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:54.822211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a256d16-258f-4876-9205-6941ccbb01b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:54.822485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a256d16-258f-4876-9205-6941ccbb01b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:54.823198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a256d16-258f-4876-9205-6941ccbb01b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:54.825641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a256d16-258f-4876-9205-6941ccbb01b5 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:54.958325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1216/2427] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:15:54 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:54.959581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff454922-6297-43a9-8817-63d0985ce096 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:54.974538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff454922-6297-43a9-8817-63d0985ce096 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:54.980719 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77e4b2-4172-4d18-8e22-5af0dae5e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:54.981603 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77e4b2-4172-4d18-8e22-5af0dae5e16c admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:54.981894 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77e4b2-4172-4d18-8e22-5af0dae5e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:54.982405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77e4b2-4172-4d18-8e22-5af0dae5e16c admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:54.983158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77e4b2-4172-4d18-8e22-5af0dae5e16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:54.986010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77e4b2-4172-4d18-8e22-5af0dae5e16c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.001563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1212/2428] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:55.006087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a256d16-258f-4876-9205-6941ccbb01b5 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.017412 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a256d16-258f-4876-9205-6941ccbb01b5 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.024440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48689b4-8100-4d24-8040-eea0eb55e951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.025704 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48689b4-8100-4d24-8040-eea0eb55e951 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.026173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48689b4-8100-4d24-8040-eea0eb55e951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.026628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48689b4-8100-4d24-8040-eea0eb55e951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.030850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48689b4-8100-4d24-8040-eea0eb55e951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.030850 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48689b4-8100-4d24-8040-eea0eb55e951 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.051221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1217/2429] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:55.055477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77e4b2-4172-4d18-8e22-5af0dae5e16c admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.068595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77e4b2-4172-4d18-8e22-5af0dae5e16c admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.074806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea643c2-d90c-4318-b01b-f29f41caca19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.076221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea643c2-d90c-4318-b01b-f29f41caca19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.076868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea643c2-d90c-4318-b01b-f29f41caca19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.077298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea643c2-d90c-4318-b01b-f29f41caca19 admin admin] PATH_INFO: `/v3/projects/ac9c7c8ccc464c0e9b35b3ce78872ee3/users/ce7463dc2cfd4ac0b7d16e9cd4cacc48/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.099994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea643c2-d90c-4318-b01b-f29f41caca19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9c7c8ccc464c0e9b35b3ce78872ee3, user_id=ce7463dc2cfd4ac0b7d16e9cd4cacc48, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.103594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea643c2-d90c-4318-b01b-f29f41caca19 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.178833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6804721-7b84-4547-a7ee-cff26cd7dc47 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.179128 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6804721-7b84-4547-a7ee-cff26cd7dc47 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.179408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6804721-7b84-4547-a7ee-cff26cd7dc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.208473 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1213/2430] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:55 2022] PUT /identity/v3/projects/ac9c7c8ccc464c0e9b35b3ce78872ee3/users/ce7463dc2cfd4ac0b7d16e9cd4cacc48/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:15:55.210269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a22f7-a80f-4f34-9bb6-8bafa4b48474 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.210560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a22f7-a80f-4f34-9bb6-8bafa4b48474 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.210838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a22f7-a80f-4f34-9bb6-8bafa4b48474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.227912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b6804721-7b84-4547-a7ee-cff26cd7dc47 None None] MFA Rules not processed for user `fe5390b9df9b445380119b995f4c457d`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:55.229210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b6804721-7b84-4547-a7ee-cff26cd7dc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:55.244407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-9c7a22f7-a80f-4f34-9bb6-8bafa4b48474 None None] MFA Rules not processed for user `ce7463dc2cfd4ac0b7d16e9cd4cacc48`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:55.245983 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9c7a22f7-a80f-4f34-9bb6-8bafa4b48474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:55.282358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:55.310061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1218/2431] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:15:55.314714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6804721-7b84-4547-a7ee-cff26cd7dc47 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.316107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b6804721-7b84-4547-a7ee-cff26cd7dc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:55.326178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1214/2432] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:55.330691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a22f7-a80f-4f34-9bb6-8bafa4b48474 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.387004 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6804721-7b84-4547-a7ee-cff26cd7dc47 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.391665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b4a2e-7117-435e-b1e4-6db86d606d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.392321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b4a2e-7117-435e-b1e4-6db86d606d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.392603 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b4a2e-7117-435e-b1e4-6db86d606d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.392945 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b4a2e-7117-435e-b1e4-6db86d606d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.393667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b4a2e-7117-435e-b1e4-6db86d606d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.394706 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a22f7-a80f-4f34-9bb6-8bafa4b48474 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.395590 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b4a2e-7117-435e-b1e4-6db86d606d53 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.399447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a66a3b-8d36-4244-a33b-7f306949ba54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.400120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a66a3b-8d36-4244-a33b-7f306949ba54 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.400403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a66a3b-8d36-4244-a33b-7f306949ba54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.400680 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a66a3b-8d36-4244-a33b-7f306949ba54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.401448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a66a3b-8d36-4244-a33b-7f306949ba54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.403290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a66a3b-8d36-4244-a33b-7f306949ba54 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.473225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1219/2433] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:15:55 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:55.477079 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b4a2e-7117-435e-b1e4-6db86d606d53 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.496642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:55.518441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1215/2434] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:15:55 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:15:55.526911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a66a3b-8d36-4244-a33b-7f306949ba54 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.549389 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b4a2e-7117-435e-b1e4-6db86d606d53 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.610090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a66a3b-8d36-4244-a33b-7f306949ba54 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.616886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d86f44-2cd8-4e0f-a3db-cd194d6e841a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.617631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d86f44-2cd8-4e0f-a3db-cd194d6e841a admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.618003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d86f44-2cd8-4e0f-a3db-cd194d6e841a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.618351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d86f44-2cd8-4e0f-a3db-cd194d6e841a admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.619100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d86f44-2cd8-4e0f-a3db-cd194d6e841a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.621266 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d86f44-2cd8-4e0f-a3db-cd194d6e841a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.626409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbec074-0401-478d-8b7b-e5d12febdbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.626409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbec074-0401-478d-8b7b-e5d12febdbef admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.626409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbec074-0401-478d-8b7b-e5d12febdbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.626409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbec074-0401-478d-8b7b-e5d12febdbef admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.626809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbec074-0401-478d-8b7b-e5d12febdbef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.626906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbec074-0401-478d-8b7b-e5d12febdbef admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.782615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1216/2435] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:15:55 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:55.786387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d86f44-2cd8-4e0f-a3db-cd194d6e841a admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.796695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d86f44-2cd8-4e0f-a3db-cd194d6e841a admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.801640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf0bc9-a7af-41f2-9b06-d7c613200bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.802915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf0bc9-a7af-41f2-9b06-d7c613200bce admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.803195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf0bc9-a7af-41f2-9b06-d7c613200bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.803468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf0bc9-a7af-41f2-9b06-d7c613200bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.804187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf0bc9-a7af-41f2-9b06-d7c613200bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.805837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf0bc9-a7af-41f2-9b06-d7c613200bce admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.818678 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1217/2436] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:15:55.822702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf0bc9-a7af-41f2-9b06-d7c613200bce admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.831728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1220/2437] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:15:55 2022] POST /identity/v3/users => generated 293 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:15:55.835836 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbec074-0401-478d-8b7b-e5d12febdbef admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.837794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf0bc9-a7af-41f2-9b06-d7c613200bce admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.843677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930f7d8-bda2-4869-8eea-8f1c98e71c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.844559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930f7d8-bda2-4869-8eea-8f1c98e71c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.845017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930f7d8-bda2-4869-8eea-8f1c98e71c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.845299 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930f7d8-bda2-4869-8eea-8f1c98e71c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.846059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930f7d8-bda2-4869-8eea-8f1c98e71c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.847305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbec074-0401-478d-8b7b-e5d12febdbef admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.847808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930f7d8-bda2-4869-8eea-8f1c98e71c56 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.852914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfb48-d3d8-4c39-ad95-0ccdf2264740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.853588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfb48-d3d8-4c39-ad95-0ccdf2264740 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.853995 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfb48-d3d8-4c39-ad95-0ccdf2264740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.854279 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfb48-d3d8-4c39-ad95-0ccdf2264740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.855013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedfb48-d3d8-4c39-ad95-0ccdf2264740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.856645 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedfb48-d3d8-4c39-ad95-0ccdf2264740 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.860976 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1218/2438] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:55.865794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930f7d8-bda2-4869-8eea-8f1c98e71c56 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.871235 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1221/2439] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:55.875719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfb48-d3d8-4c39-ad95-0ccdf2264740 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:55.876507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930f7d8-bda2-4869-8eea-8f1c98e71c56 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.882819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc53c8-83b5-42c3-87ae-a2e0f6edf722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.886575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc53c8-83b5-42c3-87ae-a2e0f6edf722 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.886575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc53c8-83b5-42c3-87ae-a2e0f6edf722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.886575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc53c8-83b5-42c3-87ae-a2e0f6edf722 admin admin] PATH_INFO: `/v3/projects/9027e179f1da4924883fab4bef6bd4a8/users/047ea8c3c50b4519b8ed3236b15180ff/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.887358 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfb48-d3d8-4c39-ad95-0ccdf2264740 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:55.899304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2bd17-cb16-4d8a-aa0f-a027526d13de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:55.900212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2bd17-cb16-4d8a-aa0f-a027526d13de admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:55.900494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2bd17-cb16-4d8a-aa0f-a027526d13de admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:55.900891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2bd17-cb16-4d8a-aa0f-a027526d13de admin admin] PATH_INFO: `/v3/projects/029709f656f44d5d999793a45eef802c/users/ab59bfdbbf9b4e48a4cee3b9b85006e2/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:55.901027 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc53c8-83b5-42c3-87ae-a2e0f6edf722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9027e179f1da4924883fab4bef6bd4a8, user_id=047ea8c3c50b4519b8ed3236b15180ff, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.903519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc53c8-83b5-42c3-87ae-a2e0f6edf722 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.922078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2bd17-cb16-4d8a-aa0f-a027526d13de admin admin] RBAC: Authorizing `identity:create_grant(project_id=029709f656f44d5d999793a45eef802c, user_id=ab59bfdbbf9b4e48a4cee3b9b85006e2, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:55.924823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2bd17-cb16-4d8a-aa0f-a027526d13de admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:55.998680 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:56.003469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1219/2440] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:55 2022] PUT /identity/v3/projects/9027e179f1da4924883fab4bef6bd4a8/users/047ea8c3c50b4519b8ed3236b15180ff/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:56.013612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d4553-aa6e-4b70-9d68-12acf3c1d1b9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:56.013937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d4553-aa6e-4b70-9d68-12acf3c1d1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:56.014221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d4553-aa6e-4b70-9d68-12acf3c1d1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:56.014919 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1222/2441] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:55 2022] PUT /identity/v3/projects/029709f656f44d5d999793a45eef802c/users/ab59bfdbbf9b4e48a4cee3b9b85006e2/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:56.018666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2bd17-cb16-4d8a-aa0f-a027526d13de admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:56.043862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c97d4553-aa6e-4b70-9d68-12acf3c1d1b9 None None] MFA Rules not processed for user `047ea8c3c50b4519b8ed3236b15180ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:56.045115 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c97d4553-aa6e-4b70-9d68-12acf3c1d1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:56.104910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2bd17-cb16-4d8a-aa0f-a027526d13de admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:56.110729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d7e0c-d7dc-4bc0-bf28-442e535e8364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:56.111730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d7e0c-d7dc-4bc0-bf28-442e535e8364 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:56.111915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d7e0c-d7dc-4bc0-bf28-442e535e8364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:56.112121 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d7e0c-d7dc-4bc0-bf28-442e535e8364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:56.112906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d7e0c-d7dc-4bc0-bf28-442e535e8364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:56.114598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d7e0c-d7dc-4bc0-bf28-442e535e8364 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:56.131047 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1223/2442] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:15:56.132164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d7e0c-d7dc-4bc0-bf28-442e535e8364 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:56.142766 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d7e0c-d7dc-4bc0-bf28-442e535e8364 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:56.148838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a0c47-22b7-4a87-b7ce-3900b387a49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:56.149853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a0c47-22b7-4a87-b7ce-3900b387a49f admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:56.150137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a0c47-22b7-4a87-b7ce-3900b387a49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:56.150415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a0c47-22b7-4a87-b7ce-3900b387a49f admin admin] PATH_INFO: `/v3/projects/029709f656f44d5d999793a45eef802c/users/ab59bfdbbf9b4e48a4cee3b9b85006e2/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:56.152524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a0c47-22b7-4a87-b7ce-3900b387a49f admin admin] RBAC: Authorizing `identity:create_grant(project_id=029709f656f44d5d999793a45eef802c, user_id=ab59bfdbbf9b4e48a4cee3b9b85006e2, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:56.153906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1220/2443] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:15:56.155538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a0c47-22b7-4a87-b7ce-3900b387a49f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:56.158081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d4553-aa6e-4b70-9d68-12acf3c1d1b9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:56.163742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:56.169297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1224/2444] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:56 2022] PUT /identity/v3/projects/029709f656f44d5d999793a45eef802c/users/ab59bfdbbf9b4e48a4cee3b9b85006e2/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:56.169610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d4553-aa6e-4b70-9d68-12acf3c1d1b9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:56.235694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c26834-9af9-47f9-a756-a1f050a28464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:56.236381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c26834-9af9-47f9-a756-a1f050a28464 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:56.236739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c26834-9af9-47f9-a756-a1f050a28464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:56.237126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c26834-9af9-47f9-a756-a1f050a28464 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:56.237828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c26834-9af9-47f9-a756-a1f050a28464 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:56.239752 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c26834-9af9-47f9-a756-a1f050a28464 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:56.341014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1221/2445] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:15:56 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:15:56.345090 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a0c47-22b7-4a87-b7ce-3900b387a49f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:56.357372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a0c47-22b7-4a87-b7ce-3900b387a49f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:56.423215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280089b-8c13-4d3b-80d1-9397a4035524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:56.423926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280089b-8c13-4d3b-80d1-9397a4035524 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:56.425159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280089b-8c13-4d3b-80d1-9397a4035524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:56.425159 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280089b-8c13-4d3b-80d1-9397a4035524 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:56.425627 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280089b-8c13-4d3b-80d1-9397a4035524 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:56.428339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0280089b-8c13-4d3b-80d1-9397a4035524 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:56.531854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1225/2446] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:15:56 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:15:56.533211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c26834-9af9-47f9-a756-a1f050a28464 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:56.546372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c26834-9af9-47f9-a756-a1f050a28464 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:56.558081 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e346cd-fb28-44be-af55-ca3539832c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:56.559548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e346cd-fb28-44be-af55-ca3539832c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:56.559548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e346cd-fb28-44be-af55-ca3539832c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:56.559548 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e346cd-fb28-44be-af55-ca3539832c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:56.560059 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e346cd-fb28-44be-af55-ca3539832c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:56.561716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e346cd-fb28-44be-af55-ca3539832c67 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:56.577083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1222/2447] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:56.578459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280089b-8c13-4d3b-80d1-9397a4035524 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:56.588896 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280089b-8c13-4d3b-80d1-9397a4035524 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:56.593875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83147b-d0f6-4dfe-8494-28f8db839dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:56.594901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83147b-d0f6-4dfe-8494-28f8db839dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:56.595401 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83147b-d0f6-4dfe-8494-28f8db839dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:56.595751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83147b-d0f6-4dfe-8494-28f8db839dbe admin admin] PATH_INFO: `/v3/projects/e07c4ead8e974c9eacdc71b4d575fe6e/users/95ec3920e5054f5aa01e8e3e617110ac/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:56.614227 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83147b-d0f6-4dfe-8494-28f8db839dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07c4ead8e974c9eacdc71b4d575fe6e, user_id=95ec3920e5054f5aa01e8e3e617110ac, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:56.617669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83147b-d0f6-4dfe-8494-28f8db839dbe admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:56.712994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1226/2448] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:56 2022] PUT /identity/v3/projects/e07c4ead8e974c9eacdc71b4d575fe6e/users/95ec3920e5054f5aa01e8e3e617110ac/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:56.717321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e346cd-fb28-44be-af55-ca3539832c67 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:56.729695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e346cd-fb28-44be-af55-ca3539832c67 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:56.793064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac1b51-1ac7-4d7e-b847-43c004c1c2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:56.793748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac1b51-1ac7-4d7e-b847-43c004c1c2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:56.794029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac1b51-1ac7-4d7e-b847-43c004c1c2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:56.794308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac1b51-1ac7-4d7e-b847-43c004c1c2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:56.795046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac1b51-1ac7-4d7e-b847-43c004c1c2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:56.796646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac1b51-1ac7-4d7e-b847-43c004c1c2d4 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:56.803418 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:56.809359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1223/2449] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:15:56.813969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83147b-d0f6-4dfe-8494-28f8db839dbe admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:56.826953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83147b-d0f6-4dfe-8494-28f8db839dbe admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:56.831907 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb68d4b-b06d-493c-906f-9719e174f2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:56.832969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb68d4b-b06d-493c-906f-9719e174f2e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:56.834028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb68d4b-b06d-493c-906f-9719e174f2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:56.834392 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb68d4b-b06d-493c-906f-9719e174f2e9 admin admin] PATH_INFO: `/v3/projects/e07c4ead8e974c9eacdc71b4d575fe6e/users/95ec3920e5054f5aa01e8e3e617110ac/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:56.836523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb68d4b-b06d-493c-906f-9719e174f2e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07c4ead8e974c9eacdc71b4d575fe6e, user_id=95ec3920e5054f5aa01e8e3e617110ac, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:56.839729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb68d4b-b06d-493c-906f-9719e174f2e9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:56.851866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1227/2450] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:15:56 2022] PUT /identity/v3/projects/e07c4ead8e974c9eacdc71b4d575fe6e/users/95ec3920e5054f5aa01e8e3e617110ac/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:15:56.867823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae96f4-d54b-4e6e-a9a9-7854482989c8 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:56.868257 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae96f4-d54b-4e6e-a9a9-7854482989c8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:56.868670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae96f4-d54b-4e6e-a9a9-7854482989c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:56.907664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-04ae96f4-d54b-4e6e-a9a9-7854482989c8 None None] MFA Rules not processed for user `95ec3920e5054f5aa01e8e3e617110ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:15:56.908970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-04ae96f4-d54b-4e6e-a9a9-7854482989c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:56.990661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1224/2451] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:15:57.002337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb68d4b-b06d-493c-906f-9719e174f2e9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:57.004154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ccb68d4b-b06d-493c-906f-9719e174f2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:57.087633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb68d4b-b06d-493c-906f-9719e174f2e9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:57.092303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9ea85-11c5-4fca-8c72-91f122d642d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:57.093253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9ea85-11c5-4fca-8c72-91f122d642d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:57.093654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9ea85-11c5-4fca-8c72-91f122d642d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:57.093880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9ea85-11c5-4fca-8c72-91f122d642d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:57.094619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae96f4-d54b-4e6e-a9a9-7854482989c8 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:57.096069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-04ae96f4-d54b-4e6e-a9a9-7854482989c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:57.106499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9ea85-11c5-4fca-8c72-91f122d642d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:57.108021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9ea85-11c5-4fca-8c72-91f122d642d3 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:57.118882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1228/2452] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:15:57.165315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae96f4-d54b-4e6e-a9a9-7854482989c8 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:57.169479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d826f99-d201-4219-9cf4-bf7cd147a123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:57.170140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d826f99-d201-4219-9cf4-bf7cd147a123 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:57.170426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d826f99-d201-4219-9cf4-bf7cd147a123 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:57.170704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d826f99-d201-4219-9cf4-bf7cd147a123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:57.172802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0d826f99-d201-4219-9cf4-bf7cd147a123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:57.263404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d826f99-d201-4219-9cf4-bf7cd147a123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:57.264907 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d826f99-d201-4219-9cf4-bf7cd147a123 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:57.278530 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1225/2453] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:15:58.467522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9ea85-11c5-4fca-8c72-91f122d642d3 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:15:58.470245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-61d9ea85-11c5-4fca-8c72-91f122d642d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:15:58.486315 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9ea85-11c5-4fca-8c72-91f122d642d3 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:15:58.500798 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ee47b-98fe-4d37-8762-49a5b004b842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:15:58.501509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ee47b-98fe-4d37-8762-49a5b004b842 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:15:58.501801 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ee47b-98fe-4d37-8762-49a5b004b842 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:15:58.502083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ee47b-98fe-4d37-8762-49a5b004b842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:15:58.631153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:15:58.642137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153ee47b-98fe-4d37-8762-49a5b004b842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:15:58.644069 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153ee47b-98fe-4d37-8762-49a5b004b842 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:15:58.666540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1229/2454] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:15:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:02.507350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d826f99-d201-4219-9cf4-bf7cd147a123 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:02.509135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0d826f99-d201-4219-9cf4-bf7cd147a123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:02.528113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d826f99-d201-4219-9cf4-bf7cd147a123 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:02.534799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:02.536144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:02.536707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:02.537331 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] PATH_INFO: `/v3/users/cb0474d4e6dd4d2b9918fd0ebe803dcd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:02.539038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb0474d4e6dd4d2b9918fd0ebe803dcd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:02.541709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:02.783275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:02.799061 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb0474d4e6dd4d2b9918fd0ebe803dcd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:02.807381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb0474d4e6dd4d2b9918fd0ebe803dcd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:02.808098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb0474d4e6dd4d2b9918fd0ebe803dcd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:02.823232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb0474d4e6dd4d2b9918fd0ebe803dcd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:02.824206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0e1ebcdb-8052-40d9-9720-5c5875d526c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb0474d4e6dd4d2b9918fd0ebe803dcd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:02.936259 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1226/2455] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:02 2022] DELETE /identity/v3/users/cb0474d4e6dd4d2b9918fd0ebe803dcd => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:03.252058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ee47b-98fe-4d37-8762-49a5b004b842 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:03.270276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ee47b-98fe-4d37-8762-49a5b004b842 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:03.368212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:03.369130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:03.369415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:03.369702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] PATH_INFO: `/v3/projects/b6acb89542314578839f701251add904` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:03.370952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6acb89542314578839f701251add904)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:03.373065 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:03.515903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6acb89542314578839f701251add904', 'request_id': 'req-89039814-8039-4d41-9fb1-9390f09df827'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:03.524223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6acb89542314578839f701251add904', 'request_id': 'req-89039814-8039-4d41-9fb1-9390f09df827'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:03.532265 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6acb89542314578839f701251add904', 'request_id': 'req-89039814-8039-4d41-9fb1-9390f09df827'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:03.650607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] The token cache is being invalidate because project b6acb89542314578839f701251add904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:03.650920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:03.651545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-89039814-8039-4d41-9fb1-9390f09df827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:03.653264 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1230/2456] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:03 2022] DELETE /identity/v3/projects/b6acb89542314578839f701251add904 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:03.658551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6c00-f8b6-4b54-987c-f05c436f7b24 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:03.659054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6c00-f8b6-4b54-987c-f05c436f7b24 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:03.659136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6c00-f8b6-4b54-987c-f05c436f7b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:03.679695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4a6c6c00-f8b6-4b54-987c-f05c436f7b24 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:03.687392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4a6c6c00-f8b6-4b54-987c-f05c436f7b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:03.757236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1227/2457] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:03.765367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97e672-29e9-4ee0-b2b6-3f451f274a56 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:03.765656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97e672-29e9-4ee0-b2b6-3f451f274a56 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:03.765960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97e672-29e9-4ee0-b2b6-3f451f274a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:03.792569 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5e97e672-29e9-4ee0-b2b6-3f451f274a56 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:03.793926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5e97e672-29e9-4ee0-b2b6-3f451f274a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:03.802403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1231/2458] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:03.807139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c6c00-f8b6-4b54-987c-f05c436f7b24 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:03.818472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c6c00-f8b6-4b54-987c-f05c436f7b24 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:03.823921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0982fb7-02bf-4e28-a54a-dbf17e8bd5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:03.824974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0982fb7-02bf-4e28-a54a-dbf17e8bd5dc admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:03.825458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0982fb7-02bf-4e28-a54a-dbf17e8bd5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:03.825738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0982fb7-02bf-4e28-a54a-dbf17e8bd5dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:03.826491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0982fb7-02bf-4e28-a54a-dbf17e8bd5dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:03.826796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0982fb7-02bf-4e28-a54a-dbf17e8bd5dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:03.831101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0982fb7-02bf-4e28-a54a-dbf17e8bd5dc admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:03.842572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1228/2459] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:16:03.847209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97e672-29e9-4ee0-b2b6-3f451f274a56 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:03.858341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97e672-29e9-4ee0-b2b6-3f451f274a56 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:03.866309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827f598-43be-4e13-89ef-7ad76d4dd084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:03.866997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827f598-43be-4e13-89ef-7ad76d4dd084 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:03.867745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827f598-43be-4e13-89ef-7ad76d4dd084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:03.867745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827f598-43be-4e13-89ef-7ad76d4dd084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:03.868232 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827f598-43be-4e13-89ef-7ad76d4dd084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:03.870229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827f598-43be-4e13-89ef-7ad76d4dd084 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:03.902883 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9982d7-e5b7-4376-ab87-2d64e81018d9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:03.903204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9982d7-e5b7-4376-ab87-2d64e81018d9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:03.903767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9982d7-e5b7-4376-ab87-2d64e81018d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:03.933972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1c9982d7-e5b7-4376-ab87-2d64e81018d9 None None] MFA Rules not processed for user `ab59bfdbbf9b4e48a4cee3b9b85006e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:03.937471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1c9982d7-e5b7-4376-ab87-2d64e81018d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:04.021877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1229/2460] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:04.026793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1232/2461] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:03 2022] POST /identity/v3/projects => generated 365 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:04.032972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9982d7-e5b7-4376-ab87-2d64e81018d9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:04.040050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827f598-43be-4e13-89ef-7ad76d4dd084 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:04.043685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4827f598-43be-4e13-89ef-7ad76d4dd084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:04.049971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9982d7-e5b7-4376-ab87-2d64e81018d9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:04.073865 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:04.136435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36271511-9895-4644-9ade-d3fc1604d644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:04.139709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36271511-9895-4644-9ade-d3fc1604d644 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:04.139709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36271511-9895-4644-9ade-d3fc1604d644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:04.139709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36271511-9895-4644-9ade-d3fc1604d644 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:04.139709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36271511-9895-4644-9ade-d3fc1604d644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:04.141114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36271511-9895-4644-9ade-d3fc1604d644 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:04.157466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:04.180533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827f598-43be-4e13-89ef-7ad76d4dd084 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:04.186463 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ae9f0-5cd8-42bb-9df7-9648929fbd50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:04.187229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ae9f0-5cd8-42bb-9df7-9648929fbd50 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:04.187593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ae9f0-5cd8-42bb-9df7-9648929fbd50 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:04.188239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ae9f0-5cd8-42bb-9df7-9648929fbd50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:04.190423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d56ae9f0-5cd8-42bb-9df7-9648929fbd50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:04.237733 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1230/2462] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:04 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:04.239058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36271511-9895-4644-9ade-d3fc1604d644 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:04.244088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-36271511-9895-4644-9ade-d3fc1604d644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:04.260969 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ae9f0-5cd8-42bb-9df7-9648929fbd50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:04.262609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ae9f0-5cd8-42bb-9df7-9648929fbd50 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:04.273421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1233/2463] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:16:04.274634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ae9f0-5cd8-42bb-9df7-9648929fbd50 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:04.285052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ae9f0-5cd8-42bb-9df7-9648929fbd50 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:04.292474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7b697-9e2d-4ceb-844a-48c25025265b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:04.293203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b697-9e2d-4ceb-844a-48c25025265b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:04.294129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b697-9e2d-4ceb-844a-48c25025265b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:04.294129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7b697-9e2d-4ceb-844a-48c25025265b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:04.294543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7b697-9e2d-4ceb-844a-48c25025265b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:04.296218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7b697-9e2d-4ceb-844a-48c25025265b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:04.307699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1234/2464] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:04.311442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7b697-9e2d-4ceb-844a-48c25025265b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:04.329495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7b697-9e2d-4ceb-844a-48c25025265b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:04.335149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1a298-9d6c-4ccc-b709-b15fe0de6442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:04.335884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1a298-9d6c-4ccc-b709-b15fe0de6442 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:04.336171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1a298-9d6c-4ccc-b709-b15fe0de6442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:04.336449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1a298-9d6c-4ccc-b709-b15fe0de6442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:04.337295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1a298-9d6c-4ccc-b709-b15fe0de6442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:04.338787 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36271511-9895-4644-9ade-d3fc1604d644 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:04.338938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1a298-9d6c-4ccc-b709-b15fe0de6442 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:04.345120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c977663-f684-4916-bf19-71546d003c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:04.346019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c977663-f684-4916-bf19-71546d003c6c service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:04.346463 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c977663-f684-4916-bf19-71546d003c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:04.349427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c977663-f684-4916-bf19-71546d003c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:04.349827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1c977663-f684-4916-bf19-71546d003c6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:04.352574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1235/2465] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:04.356892 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1a298-9d6c-4ccc-b709-b15fe0de6442 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:04.368290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1a298-9d6c-4ccc-b709-b15fe0de6442 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:04.374080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ed11f-3cf1-4fa8-9818-d9da61a6bb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:04.375063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ed11f-3cf1-4fa8-9818-d9da61a6bb8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:04.375484 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ed11f-3cf1-4fa8-9818-d9da61a6bb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:04.375831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ed11f-3cf1-4fa8-9818-d9da61a6bb8c admin admin] PATH_INFO: `/v3/projects/bb8be56d9da940889dd825563ae7ab65/users/ff001e4d405d4d279479430d4b7291e0/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:04.396707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ed11f-3cf1-4fa8-9818-d9da61a6bb8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb8be56d9da940889dd825563ae7ab65, user_id=ff001e4d405d4d279479430d4b7291e0, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:04.401057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ed11f-3cf1-4fa8-9818-d9da61a6bb8c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:04.429499 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c977663-f684-4916-bf19-71546d003c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:04.430940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c977663-f684-4916-bf19-71546d003c6c service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:04.439764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1231/2466] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:04.491207 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1236/2467] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:04 2022] PUT /identity/v3/projects/bb8be56d9da940889dd825563ae7ab65/users/ff001e4d405d4d279479430d4b7291e0/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:04.494351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9439e1-542d-4952-ae72-131a6f512250 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:04.494648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9439e1-542d-4952-ae72-131a6f512250 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:04.495005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9439e1-542d-4952-ae72-131a6f512250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:04.536404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3c9439e1-542d-4952-ae72-131a6f512250 None None] MFA Rules not processed for user `ff001e4d405d4d279479430d4b7291e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:04.537754 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3c9439e1-542d-4952-ae72-131a6f512250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:04.601046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:04.611682 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1232/2468] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:04.616354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ed11f-3cf1-4fa8-9818-d9da61a6bb8c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:04.682470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ed11f-3cf1-4fa8-9818-d9da61a6bb8c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:04.687160 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d74a40-820c-43cc-b683-590652df37c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:04.687816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d74a40-820c-43cc-b683-590652df37c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:04.688108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d74a40-820c-43cc-b683-590652df37c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:04.688385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d74a40-820c-43cc-b683-590652df37c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:04.689113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d74a40-820c-43cc-b683-590652df37c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:04.692409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d74a40-820c-43cc-b683-590652df37c8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:04.804872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1237/2469] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:04 2022] POST /identity/v3/projects => generated 363 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:04.809004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9439e1-542d-4952-ae72-131a6f512250 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:04.820720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9439e1-542d-4952-ae72-131a6f512250 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:04.879361 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abd9d1-fb1c-4960-82c4-8fd7e343e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:04.880575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abd9d1-fb1c-4960-82c4-8fd7e343e87e admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:04.881599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abd9d1-fb1c-4960-82c4-8fd7e343e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:04.881874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abd9d1-fb1c-4960-82c4-8fd7e343e87e admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:04.882574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abd9d1-fb1c-4960-82c4-8fd7e343e87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:04.884531 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abd9d1-fb1c-4960-82c4-8fd7e343e87e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:04.994467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1233/2470] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:04 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:04.998273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d74a40-820c-43cc-b683-590652df37c8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:05.009158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d74a40-820c-43cc-b683-590652df37c8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:05.013730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f61e96-88aa-4fee-8fac-9b74efec3755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:05.014381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f61e96-88aa-4fee-8fac-9b74efec3755 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:05.014665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f61e96-88aa-4fee-8fac-9b74efec3755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:05.014943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f61e96-88aa-4fee-8fac-9b74efec3755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:05.015671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f61e96-88aa-4fee-8fac-9b74efec3755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:05.017316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f61e96-88aa-4fee-8fac-9b74efec3755 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:05.028234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1238/2471] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:05.032464 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abd9d1-fb1c-4960-82c4-8fd7e343e87e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:05.042851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abd9d1-fb1c-4960-82c4-8fd7e343e87e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:05.047459 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813dfa5-17e8-4a3d-9e9f-a8afe680a0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:05.048324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813dfa5-17e8-4a3d-9e9f-a8afe680a0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:05.048609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813dfa5-17e8-4a3d-9e9f-a8afe680a0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:05.048923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813dfa5-17e8-4a3d-9e9f-a8afe680a0eb admin admin] PATH_INFO: `/v3/projects/5f3b3db69cb5497d870b209df02e0dfa/users/cdd13923c77a4b7982a4d6925a25b242/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:05.065801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813dfa5-17e8-4a3d-9e9f-a8afe680a0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3b3db69cb5497d870b209df02e0dfa, user_id=cdd13923c77a4b7982a4d6925a25b242, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:05.069414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813dfa5-17e8-4a3d-9e9f-a8afe680a0eb admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:05.151431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f61e96-88aa-4fee-8fac-9b74efec3755 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:05.151909 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1234/2472] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:05 2022] PUT /identity/v3/projects/5f3b3db69cb5497d870b209df02e0dfa/users/cdd13923c77a4b7982a4d6925a25b242/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:05.163273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f61e96-88aa-4fee-8fac-9b74efec3755 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:05.181269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:05.220474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d651cf-c272-4da9-9985-79920a0d4782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:05.221173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d651cf-c272-4da9-9985-79920a0d4782 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:05.221457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d651cf-c272-4da9-9985-79920a0d4782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:05.221735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d651cf-c272-4da9-9985-79920a0d4782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:05.222459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d651cf-c272-4da9-9985-79920a0d4782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:05.224055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d651cf-c272-4da9-9985-79920a0d4782 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:05.234241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1239/2473] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:05.238724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813dfa5-17e8-4a3d-9e9f-a8afe680a0eb admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:05.249437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813dfa5-17e8-4a3d-9e9f-a8afe680a0eb admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:05.254176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265738d3-4ea0-4399-8151-3ee85fad6b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:05.255615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265738d3-4ea0-4399-8151-3ee85fad6b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:05.255903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265738d3-4ea0-4399-8151-3ee85fad6b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:05.256177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265738d3-4ea0-4399-8151-3ee85fad6b78 admin admin] PATH_INFO: `/v3/projects/5f3b3db69cb5497d870b209df02e0dfa/users/cdd13923c77a4b7982a4d6925a25b242/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:05.258195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265738d3-4ea0-4399-8151-3ee85fad6b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3b3db69cb5497d870b209df02e0dfa, user_id=cdd13923c77a4b7982a4d6925a25b242, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:05.260867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265738d3-4ea0-4399-8151-3ee85fad6b78 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:05.279096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1235/2474] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:05 2022] PUT /identity/v3/projects/5f3b3db69cb5497d870b209df02e0dfa/users/cdd13923c77a4b7982a4d6925a25b242/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:05.291049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c0c7c-2d4d-4c06-9ffa-455b23fe44f2 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:05.291572 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c0c7c-2d4d-4c06-9ffa-455b23fe44f2 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:05.291657 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c0c7c-2d4d-4c06-9ffa-455b23fe44f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:05.319932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-079c0c7c-2d4d-4c06-9ffa-455b23fe44f2 None None] MFA Rules not processed for user `cdd13923c77a4b7982a4d6925a25b242`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:05.321243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-079c0c7c-2d4d-4c06-9ffa-455b23fe44f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:05.398894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1240/2475] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:05.409928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265738d3-4ea0-4399-8151-3ee85fad6b78 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:05.412099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-265738d3-4ea0-4399-8151-3ee85fad6b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:05.472519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265738d3-4ea0-4399-8151-3ee85fad6b78 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:05.477281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8663a6b-e573-444b-a3f6-ae03bc64d54a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:05.478035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8663a6b-e573-444b-a3f6-ae03bc64d54a service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:05.478382 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8663a6b-e573-444b-a3f6-ae03bc64d54a service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:05.478663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8663a6b-e573-444b-a3f6-ae03bc64d54a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:05.489217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8663a6b-e573-444b-a3f6-ae03bc64d54a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:05.491392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8663a6b-e573-444b-a3f6-ae03bc64d54a service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:05.502417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1236/2476] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:05 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:05.747062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c0c7c-2d4d-4c06-9ffa-455b23fe44f2 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:05.757292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c0c7c-2d4d-4c06-9ffa-455b23fe44f2 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:05.762059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebba08-2192-4d34-805a-d7ac1b052f30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:05.762993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebba08-2192-4d34-805a-d7ac1b052f30 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:05.763840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebba08-2192-4d34-805a-d7ac1b052f30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:05.763840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebba08-2192-4d34-805a-d7ac1b052f30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:05.829127 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebba08-2192-4d34-805a-d7ac1b052f30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:05.830582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebba08-2192-4d34-805a-d7ac1b052f30 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:05.840695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1241/2477] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:07.086292 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09286d3-dc43-4368-b900-471aad7800d8 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:07.086803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09286d3-dc43-4368-b900-471aad7800d8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:07.086886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09286d3-dc43-4368-b900-471aad7800d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:07.106260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c09286d3-dc43-4368-b900-471aad7800d8 None None] MFA Rules not processed for user `fe5390b9df9b445380119b995f4c457d`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:07.107557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c09286d3-dc43-4368-b900-471aad7800d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:07.157239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:07.168652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1237/2478] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:09.049937 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebba08-2192-4d34-805a-d7ac1b052f30 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:09.051431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-23ebba08-2192-4d34-805a-d7ac1b052f30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:09.097755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:09.120425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebba08-2192-4d34-805a-d7ac1b052f30 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:09.125104 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:09.125979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:09.126259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:09.126531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] PATH_INFO: `/v3/users/ce7463dc2cfd4ac0b7d16e9cd4cacc48` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:09.127788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7463dc2cfd4ac0b7d16e9cd4cacc48)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:09.129816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:09.342002 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7463dc2cfd4ac0b7d16e9cd4cacc48'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:09.349618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7463dc2cfd4ac0b7d16e9cd4cacc48'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:09.350272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7463dc2cfd4ac0b7d16e9cd4cacc48'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:09.350910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7463dc2cfd4ac0b7d16e9cd4cacc48'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:09.363495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7463dc2cfd4ac0b7d16e9cd4cacc48'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:09.465895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1242/2479] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:09 2022] DELETE /identity/v3/users/ce7463dc2cfd4ac0b7d16e9cd4cacc48 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:09.470330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c09a25-9626-47b5-9c97-f21d98a90160 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:09.470801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c09a25-9626-47b5-9c97-f21d98a90160 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:09.470894 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c09a25-9626-47b5-9c97-f21d98a90160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:09.489911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b0c09a25-9626-47b5-9c97-f21d98a90160 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:09.491113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b0c09a25-9626-47b5-9c97-f21d98a90160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:09.552563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1238/2480] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:09.562185 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:09.563596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:09.626300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a15c61-080e-41c6-aedc-a01ec8ac2140 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:09.628395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2fa53-e77c-4437-aabc-d5891a6f872b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:09.629070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2fa53-e77c-4437-aabc-d5891a6f872b service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:09.629353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2fa53-e77c-4437-aabc-d5891a6f872b service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:09.629627 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2fa53-e77c-4437-aabc-d5891a6f872b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:09.638806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2fa53-e77c-4437-aabc-d5891a6f872b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:09.640237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2fa53-e77c-4437-aabc-d5891a6f872b service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:09.651968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1243/2481] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:10.117454 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c09a25-9626-47b5-9c97-f21d98a90160 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:10.118931 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b0c09a25-9626-47b5-9c97-f21d98a90160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:10.133362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c09a25-9626-47b5-9c97-f21d98a90160 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:10.134567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2fa53-e77c-4437-aabc-d5891a6f872b service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:10.136070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-82e2fa53-e77c-4437-aabc-d5891a6f872b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:10.139397 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:10.140403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:10.140759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:10.141074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] PATH_INFO: `/v3/users/ff001e4d405d4d279479430d4b7291e0` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:10.142273 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff001e4d405d4d279479430d4b7291e0)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:10.144338 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:10.151329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2fa53-e77c-4437-aabc-d5891a6f872b service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:10.162811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:10.164181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:10.165396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:10.165949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] PATH_INFO: `/v3/projects/ac9c7c8ccc464c0e9b35b3ce78872ee3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:10.167744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9c7c8ccc464c0e9b35b3ce78872ee3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:10.170729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:10.380640 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff001e4d405d4d279479430d4b7291e0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.380977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9c7c8ccc464c0e9b35b3ce78872ee3', 'request_id': 'req-250bfbec-44af-460c-8621-374f31a75eaa'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.386612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:10.388934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9c7c8ccc464c0e9b35b3ce78872ee3', 'request_id': 'req-250bfbec-44af-460c-8621-374f31a75eaa'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.389385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff001e4d405d4d279479430d4b7291e0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.390249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff001e4d405d4d279479430d4b7291e0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.395752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9c7c8ccc464c0e9b35b3ce78872ee3', 'request_id': 'req-250bfbec-44af-460c-8621-374f31a75eaa'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.406687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff001e4d405d4d279479430d4b7291e0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.406687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-cd8e766f-9919-4038-964c-3cb11c2eb194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff001e4d405d4d279479430d4b7291e0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.465186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] The token cache is being invalidate because project ac9c7c8ccc464c0e9b35b3ce78872ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:10.465509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.466129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.467702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1244/2482] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:10 2022] DELETE /identity/v3/projects/ac9c7c8ccc464c0e9b35b3ce78872ee3 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:10.471501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:10.472795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:10.540117 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bfbec-44af-460c-8621-374f31a75eaa admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:10.547615 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1239/2483] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:10 2022] DELETE /identity/v3/users/ff001e4d405d4d279479430d4b7291e0 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:10.548214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:10.548214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:10.548214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:10.548214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] PATH_INFO: `/v3/users/047ea8c3c50b4519b8ed3236b15180ff` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:10.548571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=047ea8c3c50b4519b8ed3236b15180ff)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:10.549799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e3deb-7f83-4995-b72d-9855fba46769 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:10.550089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:10.550222 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e3deb-7f83-4995-b72d-9855fba46769 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:10.550440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e3deb-7f83-4995-b72d-9855fba46769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:10.569186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-a05e3deb-7f83-4995-b72d-9855fba46769 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:10.570409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a05e3deb-7f83-4995-b72d-9855fba46769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:10.637587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1240/2484] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:10.648104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e3deb-7f83-4995-b72d-9855fba46769 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:10.649538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a05e3deb-7f83-4995-b72d-9855fba46769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:10.658970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:10.706974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e3deb-7f83-4995-b72d-9855fba46769 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:10.711348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14cb3f-0e52-48d6-802b-80d9573ffd8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:10.711992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14cb3f-0e52-48d6-802b-80d9573ffd8a service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:10.712274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14cb3f-0e52-48d6-802b-80d9573ffd8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:10.712550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14cb3f-0e52-48d6-802b-80d9573ffd8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:10.721613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14cb3f-0e52-48d6-802b-80d9573ffd8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:10.723176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14cb3f-0e52-48d6-802b-80d9573ffd8a service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:10.732645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1241/2485] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:16:10.757656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047ea8c3c50b4519b8ed3236b15180ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.775172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047ea8c3c50b4519b8ed3236b15180ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.775927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047ea8c3c50b4519b8ed3236b15180ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.776528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047ea8c3c50b4519b8ed3236b15180ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.789861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047ea8c3c50b4519b8ed3236b15180ff'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:10.907879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1245/2486] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:10 2022] DELETE /identity/v3/users/047ea8c3c50b4519b8ed3236b15180ff => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:11.201359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14cb3f-0e52-48d6-802b-80d9573ffd8a service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:11.202790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2b14cb3f-0e52-48d6-802b-80d9573ffd8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:11.215724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14cb3f-0e52-48d6-802b-80d9573ffd8a service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:11.285017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:11.286153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:11.286471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:11.286726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] PATH_INFO: `/v3/projects/bb8be56d9da940889dd825563ae7ab65` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:11.288613 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb8be56d9da940889dd825563ae7ab65)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:11.290465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:11.359272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:11.361149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:11.376640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05bd7e-d44c-4a2e-a8c4-90529896c411 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:11.381931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:11.382947 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:11.383306 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:11.383587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] PATH_INFO: `/v3/projects/9027e179f1da4924883fab4bef6bd4a8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:11.385001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9027e179f1da4924883fab4bef6bd4a8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:11.386857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:11.454024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb8be56d9da940889dd825563ae7ab65', 'request_id': 'req-f999f6bf-abac-4036-b96c-4d530829608e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:11.460625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb8be56d9da940889dd825563ae7ab65', 'request_id': 'req-f999f6bf-abac-4036-b96c-4d530829608e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:11.959244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9027e179f1da4924883fab4bef6bd4a8', 'request_id': 'req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:11.964699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9027e179f1da4924883fab4bef6bd4a8', 'request_id': 'req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:11.965811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:11.970816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb8be56d9da940889dd825563ae7ab65', 'request_id': 'req-f999f6bf-abac-4036-b96c-4d530829608e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:11.972461 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9027e179f1da4924883fab4bef6bd4a8', 'request_id': 'req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:11.988144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] The token cache is being invalidate because project bb8be56d9da940889dd825563ae7ab65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:11.988560 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:11.989250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:11.991978 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1242/2487] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:11 2022] DELETE /identity/v3/projects/bb8be56d9da940889dd825563ae7ab65 => generated 0 bytes in 791 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:11.995633 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:11.997057 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:12.039848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:12.062151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999f6bf-abac-4036-b96c-4d530829608e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:12.117964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:12.118844 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:12.119129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:12.119407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] PATH_INFO: `/v3/users/cdd13923c77a4b7982a4d6925a25b242` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:12.120625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdd13923c77a4b7982a4d6925a25b242)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:12.122660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:12.175142 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] The token cache is being invalidate because project 9027e179f1da4924883fab4bef6bd4a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:12.175429 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.176035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.177708 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1246/2488] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:11 2022] DELETE /identity/v3/projects/9027e179f1da4924883fab4bef6bd4a8 => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:12.181578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:12.182975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:12.195864 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b4b4d-3a06-4e72-b1a6-cf1c8f4490e9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:12.200297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:12.201197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:12.220415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:12.220415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] PATH_INFO: `/v3/users/95ec3920e5054f5aa01e8e3e617110ac` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:12.220415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ec3920e5054f5aa01e8e3e617110ac)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:12.220415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:12.400173 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ec3920e5054f5aa01e8e3e617110ac'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.402408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdd13923c77a4b7982a4d6925a25b242'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.407988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ec3920e5054f5aa01e8e3e617110ac'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.408792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ec3920e5054f5aa01e8e3e617110ac'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.409435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ec3920e5054f5aa01e8e3e617110ac'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.409611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdd13923c77a4b7982a4d6925a25b242'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.410288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdd13923c77a4b7982a4d6925a25b242'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.422556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ec3920e5054f5aa01e8e3e617110ac'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.429301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdd13923c77a4b7982a4d6925a25b242'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.430224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdd13923c77a4b7982a4d6925a25b242'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:12.522814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1247/2489] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:12 2022] DELETE /identity/v3/users/95ec3920e5054f5aa01e8e3e617110ac => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:12.593124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1243/2490] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:11 2022] DELETE /identity/v3/users/cdd13923c77a4b7982a4d6925a25b242 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:12.981877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:12.984260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:13.068024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077c88b-101a-4a13-a1f7-5bde3abf893d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.124259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:13.124994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.125352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.125579 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] PATH_INFO: `/v3/projects/e07c4ead8e974c9eacdc71b4d575fe6e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.126432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.127039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07c4ead8e974c9eacdc71b4d575fe6e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:13.128776 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:13.129652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:13.138369 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:13.154107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f95414-84de-4562-b86b-4269061c32d7 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.159370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:13.160305 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.160654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.161069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] PATH_INFO: `/v3/projects/5f3b3db69cb5497d870b209df02e0dfa` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.162927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f3b3db69cb5497d870b209df02e0dfa)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:13.164858 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:13.262566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07c4ead8e974c9eacdc71b4d575fe6e', 'request_id': 'req-6ff53070-537a-4fb5-8067-7f42a7cf9dad'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:13.272807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07c4ead8e974c9eacdc71b4d575fe6e', 'request_id': 'req-6ff53070-537a-4fb5-8067-7f42a7cf9dad'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:13.283877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07c4ead8e974c9eacdc71b4d575fe6e', 'request_id': 'req-6ff53070-537a-4fb5-8067-7f42a7cf9dad'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:13.296363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f3b3db69cb5497d870b209df02e0dfa', 'request_id': 'req-eb10f16d-2483-4c5a-9dca-67016a2711da'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:13.302580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f3b3db69cb5497d870b209df02e0dfa', 'request_id': 'req-eb10f16d-2483-4c5a-9dca-67016a2711da'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:13.392623 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] The token cache is being invalidate because project e07c4ead8e974c9eacdc71b4d575fe6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:13.392955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:13.393571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6ff53070-537a-4fb5-8067-7f42a7cf9dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:13.395322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1248/2491] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:12 2022] DELETE /identity/v3/projects/e07c4ead8e974c9eacdc71b4d575fe6e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:13.401519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8144ff-c6ee-4607-b02f-0e807d66fc59 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.402092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8144ff-c6ee-4607-b02f-0e807d66fc59 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.402092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8144ff-c6ee-4607-b02f-0e807d66fc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.420782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-9a8144ff-c6ee-4607-b02f-0e807d66fc59 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:13.424966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9a8144ff-c6ee-4607-b02f-0e807d66fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:13.451968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f3b3db69cb5497d870b209df02e0dfa', 'request_id': 'req-eb10f16d-2483-4c5a-9dca-67016a2711da'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:13.451968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:13.463444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] The token cache is being invalidate because project 5f3b3db69cb5497d870b209df02e0dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:13.463816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:13.464597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb10f16d-2483-4c5a-9dca-67016a2711da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:13.466470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1244/2492] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:13 2022] DELETE /identity/v3/projects/5f3b3db69cb5497d870b209df02e0dfa => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:13.471993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e0d89-0984-4790-a3db-80e150fbbbbf None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.472283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e0d89-0984-4790-a3db-80e150fbbbbf None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.472567 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e0d89-0984-4790-a3db-80e150fbbbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.487445 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1249/2493] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:13.493424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-ec2e0d89-0984-4790-a3db-80e150fbbbbf None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:13.495177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ec2e0d89-0984-4790-a3db-80e150fbbbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:13.496275 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6661a9-65c6-4656-b6c6-b69dd1cc8669 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.496567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6661a9-65c6-4656-b6c6-b69dd1cc8669 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.496877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6661a9-65c6-4656-b6c6-b69dd1cc8669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.503884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1245/2494] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:13.512105 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676ae02-f2ab-44d1-b866-8127a969c3d8 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.512392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676ae02-f2ab-44d1-b866-8127a969c3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.512665 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676ae02-f2ab-44d1-b866-8127a969c3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.522076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-8d6661a9-65c6-4656-b6c6-b69dd1cc8669 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:13.523277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-8d6661a9-65c6-4656-b6c6-b69dd1cc8669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:13.530744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1676ae02-f2ab-44d1-b866-8127a969c3d8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:13.531114 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1250/2495] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:13.531902 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1676ae02-f2ab-44d1-b866-8127a969c3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:13.535105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6661a9-65c6-4656-b6c6-b69dd1cc8669 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.539989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1246/2496] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:13.546298 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676ae02-f2ab-44d1-b866-8127a969c3d8 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.548023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6661a9-65c6-4656-b6c6-b69dd1cc8669 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.553603 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e80e3a-e276-4100-90eb-03f3dcf4e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:13.556554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e80e3a-e276-4100-90eb-03f3dcf4e09c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.556554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e80e3a-e276-4100-90eb-03f3dcf4e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.556554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e80e3a-e276-4100-90eb-03f3dcf4e09c admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.556554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e80e3a-e276-4100-90eb-03f3dcf4e09c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:13.556554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e80e3a-e276-4100-90eb-03f3dcf4e09c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:13.557154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676ae02-f2ab-44d1-b866-8127a969c3d8 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.557596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e80e3a-e276-4100-90eb-03f3dcf4e09c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:13.561304 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6938a-524f-4c7c-aa42-77777a12cfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:13.561961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6938a-524f-4c7c-aa42-77777a12cfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.562269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6938a-524f-4c7c-aa42-77777a12cfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.562519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6938a-524f-4c7c-aa42-77777a12cfeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.563246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6938a-524f-4c7c-aa42-77777a12cfeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:13.563556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6938a-524f-4c7c-aa42-77777a12cfeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:13.565182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6938a-524f-4c7c-aa42-77777a12cfeb admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:13.570577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1251/2497] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:16:13.575994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e80e3a-e276-4100-90eb-03f3dcf4e09c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.581158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1247/2498] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:16:13.582459 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6938a-524f-4c7c-aa42-77777a12cfeb admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.587110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e80e3a-e276-4100-90eb-03f3dcf4e09c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.591687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6aa830-9c41-4ccf-a2b4-82fa39812450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:13.592323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aa830-9c41-4ccf-a2b4-82fa39812450 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.592532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6938a-524f-4c7c-aa42-77777a12cfeb admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.592648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aa830-9c41-4ccf-a2b4-82fa39812450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.592898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aa830-9c41-4ccf-a2b4-82fa39812450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.593571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6aa830-9c41-4ccf-a2b4-82fa39812450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:13.595445 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6aa830-9c41-4ccf-a2b4-82fa39812450 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:13.597168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e7258-4365-41c0-a9cc-cc8c53ca78af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:13.597797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e7258-4365-41c0-a9cc-cc8c53ca78af admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.598080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e7258-4365-41c0-a9cc-cc8c53ca78af admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.598359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e7258-4365-41c0-a9cc-cc8c53ca78af admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.599032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e7258-4365-41c0-a9cc-cc8c53ca78af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:13.601608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e7258-4365-41c0-a9cc-cc8c53ca78af admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:13.714883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1252/2499] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:13 2022] POST /identity/v3/projects => generated 391 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:13.718720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6aa830-9c41-4ccf-a2b4-82fa39812450 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.756324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1248/2500] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:13 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:13.757526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e7258-4365-41c0-a9cc-cc8c53ca78af admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.785346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6aa830-9c41-4ccf-a2b4-82fa39812450 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.825288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e7258-4365-41c0-a9cc-cc8c53ca78af admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.830087 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f0cda-0506-42f6-a6b1-19b19360bcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:13.830761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f0cda-0506-42f6-a6b1-19b19360bcd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.831049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f0cda-0506-42f6-a6b1-19b19360bcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.831330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f0cda-0506-42f6-a6b1-19b19360bcd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.832030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f0cda-0506-42f6-a6b1-19b19360bcd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:13.834585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f0cda-0506-42f6-a6b1-19b19360bcd5 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:13.850126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c081e-d657-47e6-a5b5-f09379f0a851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:13.850795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c081e-d657-47e6-a5b5-f09379f0a851 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.851077 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c081e-d657-47e6-a5b5-f09379f0a851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.851476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c081e-d657-47e6-a5b5-f09379f0a851 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.852039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c081e-d657-47e6-a5b5-f09379f0a851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:13.854001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c081e-d657-47e6-a5b5-f09379f0a851 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:13.946953 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1249/2501] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:13 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:13.951033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f0cda-0506-42f6-a6b1-19b19360bcd5 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.958923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1253/2502] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:13 2022] POST /identity/v3/users => generated 296 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:13.961297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f0cda-0506-42f6-a6b1-19b19360bcd5 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.962939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c081e-d657-47e6-a5b5-f09379f0a851 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.966185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76171ea6-6a71-436c-9546-e41174975830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:13.966836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76171ea6-6a71-436c-9546-e41174975830 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.967122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76171ea6-6a71-436c-9546-e41174975830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.967398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76171ea6-6a71-436c-9546-e41174975830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.968144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76171ea6-6a71-436c-9546-e41174975830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:13.969763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76171ea6-6a71-436c-9546-e41174975830 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:13.972368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c081e-d657-47e6-a5b5-f09379f0a851 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.977216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a3773-4243-4b27-b548-b0383529712c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:13.977982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a3773-4243-4b27-b548-b0383529712c admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:13.978338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a3773-4243-4b27-b548-b0383529712c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:13.978714 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a3773-4243-4b27-b548-b0383529712c admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:13.979444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a3773-4243-4b27-b548-b0383529712c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:13.980380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1250/2503] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:13.981288 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a3773-4243-4b27-b548-b0383529712c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:13.984102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76171ea6-6a71-436c-9546-e41174975830 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.993055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1254/2504] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:13.994452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76171ea6-6a71-436c-9546-e41174975830 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:13.996614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a3773-4243-4b27-b548-b0383529712c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:13.999589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14896610-9d7a-48e9-a8fe-a965c1776904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.000505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14896610-9d7a-48e9-a8fe-a965c1776904 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.000921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14896610-9d7a-48e9-a8fe-a965c1776904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.001276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14896610-9d7a-48e9-a8fe-a965c1776904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.002106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14896610-9d7a-48e9-a8fe-a965c1776904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.003862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14896610-9d7a-48e9-a8fe-a965c1776904 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.011440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a3773-4243-4b27-b548-b0383529712c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.016315 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4eeb17-e085-43a8-a3bc-18f3c5fcadba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.016731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1251/2505] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:14.017037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eeb17-e085-43a8-a3bc-18f3c5fcadba admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.017381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eeb17-e085-43a8-a3bc-18f3c5fcadba admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.017663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eeb17-e085-43a8-a3bc-18f3c5fcadba admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.018444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4eeb17-e085-43a8-a3bc-18f3c5fcadba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.020884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14896610-9d7a-48e9-a8fe-a965c1776904 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.021595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4eeb17-e085-43a8-a3bc-18f3c5fcadba admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.031482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14896610-9d7a-48e9-a8fe-a965c1776904 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.032544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1255/2506] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:14.036506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4eeb17-e085-43a8-a3bc-18f3c5fcadba admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.039921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7c06c-e829-4ef9-9fe0-e5ac22a78ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.040887 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7c06c-e829-4ef9-9fe0-e5ac22a78ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.041170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7c06c-e829-4ef9-9fe0-e5ac22a78ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.041444 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7c06c-e829-4ef9-9fe0-e5ac22a78ac3 admin admin] PATH_INFO: `/v3/projects/bce2082b12114152a0417c329b0c9a36/users/d7da49ac403146a385550f32e0fe2005/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.046689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4eeb17-e085-43a8-a3bc-18f3c5fcadba admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.051390 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b23fab-6bb7-413b-8cd7-39095a8d1a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.052261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b23fab-6bb7-413b-8cd7-39095a8d1a78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.052547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b23fab-6bb7-413b-8cd7-39095a8d1a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.052860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b23fab-6bb7-413b-8cd7-39095a8d1a78 admin admin] PATH_INFO: `/v3/projects/029eb2a96778465bab923d2c131697e9/users/038c74a63c5b4d1da2eeaf0a905f6720/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.056392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7c06c-e829-4ef9-9fe0-e5ac22a78ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce2082b12114152a0417c329b0c9a36, user_id=d7da49ac403146a385550f32e0fe2005, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.059064 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7c06c-e829-4ef9-9fe0-e5ac22a78ac3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.068385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b23fab-6bb7-413b-8cd7-39095a8d1a78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=029eb2a96778465bab923d2c131697e9, user_id=038c74a63c5b4d1da2eeaf0a905f6720, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.071669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b23fab-6bb7-413b-8cd7-39095a8d1a78 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.170056 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1252/2507] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:14 2022] PUT /identity/v3/projects/bce2082b12114152a0417c329b0c9a36/users/d7da49ac403146a385550f32e0fe2005/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:14.181296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d61fe-ad6e-45d3-aabd-f740864c0eef None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.181586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d61fe-ad6e-45d3-aabd-f740864c0eef None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.181863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d61fe-ad6e-45d3-aabd-f740864c0eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.187836 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:14.195551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1256/2508] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:14 2022] PUT /identity/v3/projects/029eb2a96778465bab923d2c131697e9/users/038c74a63c5b4d1da2eeaf0a905f6720/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:14.206689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf7dc2-bb4e-46f2-9a6e-25991721a2bb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.206914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf7dc2-bb4e-46f2-9a6e-25991721a2bb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.207249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf7dc2-bb4e-46f2-9a6e-25991721a2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.212990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-fe6d61fe-ad6e-45d3-aabd-f740864c0eef None None] MFA Rules not processed for user `d7da49ac403146a385550f32e0fe2005`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:14.220017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fe6d61fe-ad6e-45d3-aabd-f740864c0eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:14.238584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-becf7dc2-bb4e-46f2-9a6e-25991721a2bb None None] MFA Rules not processed for user `038c74a63c5b4d1da2eeaf0a905f6720`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:14.239816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-becf7dc2-bb4e-46f2-9a6e-25991721a2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:14.290025 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1253/2509] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:14.295392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d61fe-ad6e-45d3-aabd-f740864c0eef None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.314713 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1257/2510] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:14.319728 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf7dc2-bb4e-46f2-9a6e-25991721a2bb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.339005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:14.366616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d61fe-ad6e-45d3-aabd-f740864c0eef None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.371453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682e7f9-4bf3-4479-ad5c-5fe8a463fafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.372113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682e7f9-4bf3-4479-ad5c-5fe8a463fafe admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.372399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682e7f9-4bf3-4479-ad5c-5fe8a463fafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.372676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682e7f9-4bf3-4479-ad5c-5fe8a463fafe admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.373399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682e7f9-4bf3-4479-ad5c-5fe8a463fafe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.375306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682e7f9-4bf3-4479-ad5c-5fe8a463fafe admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.383359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf7dc2-bb4e-46f2-9a6e-25991721a2bb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.387922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df103ed-9372-4819-a23c-354404de02e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.388584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df103ed-9372-4819-a23c-354404de02e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.388895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df103ed-9372-4819-a23c-354404de02e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.389180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df103ed-9372-4819-a23c-354404de02e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.389869 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df103ed-9372-4819-a23c-354404de02e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.391764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df103ed-9372-4819-a23c-354404de02e0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.490489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1254/2511] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:14 2022] POST /identity/v3/projects => generated 369 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:14.497398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682e7f9-4bf3-4479-ad5c-5fe8a463fafe admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.535443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1258/2512] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:14 2022] POST /identity/v3/projects => generated 391 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:14.539879 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df103ed-9372-4819-a23c-354404de02e0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.560317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682e7f9-4bf3-4479-ad5c-5fe8a463fafe admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.606762 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df103ed-9372-4819-a23c-354404de02e0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.611985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74f15c-efc7-4c69-af80-b4c7f5c49ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.612656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74f15c-efc7-4c69-af80-b4c7f5c49ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.613071 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74f15c-efc7-4c69-af80-b4c7f5c49ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.613345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74f15c-efc7-4c69-af80-b4c7f5c49ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.614032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74f15c-efc7-4c69-af80-b4c7f5c49ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.616832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74f15c-efc7-4c69-af80-b4c7f5c49ec7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.624920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf943eef-6dd8-4eb9-9f48-01ae2bf4d809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.625761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf943eef-6dd8-4eb9-9f48-01ae2bf4d809 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.625910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf943eef-6dd8-4eb9-9f48-01ae2bf4d809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.626122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf943eef-6dd8-4eb9-9f48-01ae2bf4d809 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.626800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf943eef-6dd8-4eb9-9f48-01ae2bf4d809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.628736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf943eef-6dd8-4eb9-9f48-01ae2bf4d809 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.770439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1259/2513] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:14 2022] POST /identity/v3/users => generated 302 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:14.775285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74f15c-efc7-4c69-af80-b4c7f5c49ec7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.786431 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74f15c-efc7-4c69-af80-b4c7f5c49ec7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.791720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a224a7-23af-41ea-afea-4a17aac7cc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.792381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a224a7-23af-41ea-afea-4a17aac7cc51 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.792665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a224a7-23af-41ea-afea-4a17aac7cc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.792974 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a224a7-23af-41ea-afea-4a17aac7cc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.793705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a224a7-23af-41ea-afea-4a17aac7cc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.795342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a224a7-23af-41ea-afea-4a17aac7cc51 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.805914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1260/2514] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:14.810253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a224a7-23af-41ea-afea-4a17aac7cc51 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.817229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:14.820580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1255/2515] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:14 2022] POST /identity/v3/users => generated 291 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:14.821638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a224a7-23af-41ea-afea-4a17aac7cc51 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.826322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539e461-6859-4819-80d8-d28d790e425f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.827439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539e461-6859-4819-80d8-d28d790e425f admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.827724 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539e461-6859-4819-80d8-d28d790e425f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.827999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539e461-6859-4819-80d8-d28d790e425f admin admin] PATH_INFO: `/v3/projects/ecb0cd8fe69b4b4bad42923507c1c817/users/dcbe5d8601a34ef0bd32d0aad9c87611/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.828225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf943eef-6dd8-4eb9-9f48-01ae2bf4d809 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.838616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf943eef-6dd8-4eb9-9f48-01ae2bf4d809 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.843690 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d84a08-e12f-450d-a7e4-702ec2e45c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.844385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d84a08-e12f-450d-a7e4-702ec2e45c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.844710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539e461-6859-4819-80d8-d28d790e425f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb0cd8fe69b4b4bad42923507c1c817, user_id=dcbe5d8601a34ef0bd32d0aad9c87611, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.844934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d84a08-e12f-450d-a7e4-702ec2e45c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.845196 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d84a08-e12f-450d-a7e4-702ec2e45c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.845957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d84a08-e12f-450d-a7e4-702ec2e45c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.847487 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539e461-6859-4819-80d8-d28d790e425f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.847669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d84a08-e12f-450d-a7e4-702ec2e45c40 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.858822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1256/2516] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:14.863115 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d84a08-e12f-450d-a7e4-702ec2e45c40 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.873623 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d84a08-e12f-450d-a7e4-702ec2e45c40 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:14.875312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:14.879587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e97d68-f464-4ab2-9618-6a7a8b0bd9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:14.881080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e97d68-f464-4ab2-9618-6a7a8b0bd9cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:14.881362 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e97d68-f464-4ab2-9618-6a7a8b0bd9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:14.881635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e97d68-f464-4ab2-9618-6a7a8b0bd9cf admin admin] PATH_INFO: `/v3/projects/2aa9f8b9f72b4753b5cd826e5bfe6a5a/users/528b75d7cd774f48a612951e91d0ade3/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:14.895578 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e97d68-f464-4ab2-9618-6a7a8b0bd9cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa9f8b9f72b4753b5cd826e5bfe6a5a, user_id=528b75d7cd774f48a612951e91d0ade3, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:14.898240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e97d68-f464-4ab2-9618-6a7a8b0bd9cf admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:14.931175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1261/2517] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:14 2022] PUT /identity/v3/projects/ecb0cd8fe69b4b4bad42923507c1c817/users/dcbe5d8601a34ef0bd32d0aad9c87611/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:14.935163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539e461-6859-4819-80d8-d28d790e425f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:14.980831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1257/2518] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:14 2022] PUT /identity/v3/projects/2aa9f8b9f72b4753b5cd826e5bfe6a5a/users/528b75d7cd774f48a612951e91d0ade3/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:14.984221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e97d68-f464-4ab2-9618-6a7a8b0bd9cf admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:15.000938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539e461-6859-4819-80d8-d28d790e425f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:15.040563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:15.051903 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e97d68-f464-4ab2-9618-6a7a8b0bd9cf admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:15.057850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6996b-7815-4b51-9a72-ba443e06bc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:15.058511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6996b-7815-4b51-9a72-ba443e06bc57 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:15.058792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6996b-7815-4b51-9a72-ba443e06bc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:15.059063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6996b-7815-4b51-9a72-ba443e06bc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:15.059298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c3ffe-09cc-41e3-a036-3dcd756ef456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:15.059852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6996b-7815-4b51-9a72-ba443e06bc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:15.059966 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c3ffe-09cc-41e3-a036-3dcd756ef456 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:15.060234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c3ffe-09cc-41e3-a036-3dcd756ef456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:15.060509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c3ffe-09cc-41e3-a036-3dcd756ef456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:15.061267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c3ffe-09cc-41e3-a036-3dcd756ef456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:15.062687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6996b-7815-4b51-9a72-ba443e06bc57 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:15.062900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c3ffe-09cc-41e3-a036-3dcd756ef456 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:15.071276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1258/2519] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:15.075512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1262/2520] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:15.075652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6996b-7815-4b51-9a72-ba443e06bc57 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:15.076395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c3ffe-09cc-41e3-a036-3dcd756ef456 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:15.085460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6996b-7815-4b51-9a72-ba443e06bc57 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:15.086455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c3ffe-09cc-41e3-a036-3dcd756ef456 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:15.090142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5140c9-a24b-493a-a175-3a888ba65a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:15.095321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5140c9-a24b-493a-a175-3a888ba65a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:15.095321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5140c9-a24b-493a-a175-3a888ba65a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:15.095554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed3870-a9f1-4bb1-862a-da95fb9afb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:15.095652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5140c9-a24b-493a-a175-3a888ba65a06 admin admin] PATH_INFO: `/v3/projects/2aa9f8b9f72b4753b5cd826e5bfe6a5a/users/528b75d7cd774f48a612951e91d0ade3/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:15.095751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed3870-a9f1-4bb1-862a-da95fb9afb70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:15.095751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed3870-a9f1-4bb1-862a-da95fb9afb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:15.095751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed3870-a9f1-4bb1-862a-da95fb9afb70 admin admin] PATH_INFO: `/v3/projects/ecb0cd8fe69b4b4bad42923507c1c817/users/dcbe5d8601a34ef0bd32d0aad9c87611/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:15.096013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5140c9-a24b-493a-a175-3a888ba65a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa9f8b9f72b4753b5cd826e5bfe6a5a, user_id=528b75d7cd774f48a612951e91d0ade3, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:15.096111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed3870-a9f1-4bb1-862a-da95fb9afb70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb0cd8fe69b4b4bad42923507c1c817, user_id=dcbe5d8601a34ef0bd32d0aad9c87611, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:15.096351 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5140c9-a24b-493a-a175-3a888ba65a06 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:15.097359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed3870-a9f1-4bb1-862a-da95fb9afb70 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:15.113177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1263/2522] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:15 2022] PUT /identity/v3/projects/ecb0cd8fe69b4b4bad42923507c1c817/users/dcbe5d8601a34ef0bd32d0aad9c87611/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:15.113420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1259/2522] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:15 2022] PUT /identity/v3/projects/2aa9f8b9f72b4753b5cd826e5bfe6a5a/users/528b75d7cd774f48a612951e91d0ade3/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:15.123813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e3f00-860d-42f0-a67f-23e648d9e721 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:15.124109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e3f00-860d-42f0-a67f-23e648d9e721 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:15.124249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b991c-c6e3-4764-ab6c-482fbc69cb2b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:15.124389 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e3f00-860d-42f0-a67f-23e648d9e721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:15.124663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b991c-c6e3-4764-ab6c-482fbc69cb2b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:15.125030 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b991c-c6e3-4764-ab6c-482fbc69cb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:15.152247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-693b991c-c6e3-4764-ab6c-482fbc69cb2b None None] MFA Rules not processed for user `dcbe5d8601a34ef0bd32d0aad9c87611`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:15.152487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4c3e3f00-860d-42f0-a67f-23e648d9e721 None None] MFA Rules not processed for user `528b75d7cd774f48a612951e91d0ade3`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:15.153549 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-693b991c-c6e3-4764-ab6c-482fbc69cb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:15.153694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4c3e3f00-860d-42f0-a67f-23e648d9e721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:15.225927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1264/2523] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:15.230464 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1260/2524] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:15.238596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b991c-c6e3-4764-ab6c-482fbc69cb2b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:15.240139 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-693b991c-c6e3-4764-ab6c-482fbc69cb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:15.245455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e3f00-860d-42f0-a67f-23e648d9e721 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:15.246816 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4c3e3f00-860d-42f0-a67f-23e648d9e721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:15.280677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:15.304293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b991c-c6e3-4764-ab6c-482fbc69cb2b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:15.306028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e3f00-860d-42f0-a67f-23e648d9e721 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:15.308182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3ab7d-5fcb-413c-a96b-3f7415dc3d24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:15.308926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3ab7d-5fcb-413c-a96b-3f7415dc3d24 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:15.309239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3ab7d-5fcb-413c-a96b-3f7415dc3d24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:15.309513 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3ab7d-5fcb-413c-a96b-3f7415dc3d24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:15.310339 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a7b01-e249-4ac1-acb6-03640e6c9f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:15.311446 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7b01-e249-4ac1-acb6-03640e6c9f8f service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:15.312604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7b01-e249-4ac1-acb6-03640e6c9f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:15.313347 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7b01-e249-4ac1-acb6-03640e6c9f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:15.322294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3ab7d-5fcb-413c-a96b-3f7415dc3d24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:15.323959 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3ab7d-5fcb-413c-a96b-3f7415dc3d24 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:15.329684 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a7b01-e249-4ac1-acb6-03640e6c9f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:15.331143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a7b01-e249-4ac1-acb6-03640e6c9f8f service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:15.334605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1265/2525] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:15.342044 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1261/2526] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:15 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:16:15.379803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3ab7d-5fcb-413c-a96b-3f7415dc3d24 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:15.389225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3ab7d-5fcb-413c-a96b-3f7415dc3d24 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:15.393125 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321f52-e635-46bc-bacd-9f7fd4c7786d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:15.393781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f52-e635-46bc-bacd-9f7fd4c7786d service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:15.394184 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f52-e635-46bc-bacd-9f7fd4c7786d service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:15.394370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321f52-e635-46bc-bacd-9f7fd4c7786d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:15.425633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:15.454148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321f52-e635-46bc-bacd-9f7fd4c7786d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:15.455570 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321f52-e635-46bc-bacd-9f7fd4c7786d service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:15.465625 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1266/2527] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:18.403972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a7b01-e249-4ac1-acb6-03640e6c9f8f service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:18.467296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a7b01-e249-4ac1-acb6-03640e6c9f8f service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:18.471726 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:18.472704 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:18.473037 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:18.473325 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] PATH_INFO: `/v3/users/d7da49ac403146a385550f32e0fe2005` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:18.475107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7da49ac403146a385550f32e0fe2005)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:18.477058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:18.696048 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7da49ac403146a385550f32e0fe2005'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:18.702852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7da49ac403146a385550f32e0fe2005'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:18.703588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7da49ac403146a385550f32e0fe2005'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:18.721050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7da49ac403146a385550f32e0fe2005'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:18.721050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7da49ac403146a385550f32e0fe2005'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:18.813917 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1262/2528] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:18 2022] DELETE /identity/v3/users/d7da49ac403146a385550f32e0fe2005 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:18.818498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491cd80-f7b9-48f6-a216-73ac982191bc None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:18.818985 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491cd80-f7b9-48f6-a216-73ac982191bc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:18.819084 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491cd80-f7b9-48f6-a216-73ac982191bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:18.840384 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-f491cd80-f7b9-48f6-a216-73ac982191bc None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:18.840384 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f491cd80-f7b9-48f6-a216-73ac982191bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:18.900439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1267/2529] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:18.912794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:18.914441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:18.976056 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7d995-8565-4e7d-ae14-1ba9fbda8d77 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:18.981208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a6376-5d99-442f-8f6b-36dda9dc98d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:18.981871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a6376-5d99-442f-8f6b-36dda9dc98d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:18.982151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a6376-5d99-442f-8f6b-36dda9dc98d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:18.982426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a6376-5d99-442f-8f6b-36dda9dc98d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:18.990869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:18.993918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a6376-5d99-442f-8f6b-36dda9dc98d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:18.995353 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a6376-5d99-442f-8f6b-36dda9dc98d8 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:19.005742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1263/2530] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:19.490195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491cd80-f7b9-48f6-a216-73ac982191bc None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:19.491800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f491cd80-f7b9-48f6-a216-73ac982191bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:19.507814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491cd80-f7b9-48f6-a216-73ac982191bc None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:19.513108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:19.514342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:19.514636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:19.514962 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] PATH_INFO: `/v3/projects/bce2082b12114152a0417c329b0c9a36` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:19.516260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce2082b12114152a0417c329b0c9a36)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:19.518309 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:19.641639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce2082b12114152a0417c329b0c9a36', 'request_id': 'req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:19.646995 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce2082b12114152a0417c329b0c9a36', 'request_id': 'req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:19.653049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce2082b12114152a0417c329b0c9a36', 'request_id': 'req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:19.761828 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] The token cache is being invalidate because project bce2082b12114152a0417c329b0c9a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:19.769072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:19.769072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:19.769072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1268/2531] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:19 2022] DELETE /identity/v3/projects/bce2082b12114152a0417c329b0c9a36 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:19.769675 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a6376-5d99-442f-8f6b-36dda9dc98d8 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:19.779156 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a6376-5d99-442f-8f6b-36dda9dc98d8 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:19.835660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:19.836535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:19.836874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:19.837153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] PATH_INFO: `/v3/users/528b75d7cd774f48a612951e91d0ade3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:19.838333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=528b75d7cd774f48a612951e91d0ade3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:19.840232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:20.039727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '528b75d7cd774f48a612951e91d0ade3'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.047968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '528b75d7cd774f48a612951e91d0ade3'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.048659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '528b75d7cd774f48a612951e91d0ade3'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.061022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '528b75d7cd774f48a612951e91d0ade3'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.061651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '528b75d7cd774f48a612951e91d0ade3'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.144824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1264/2532] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:19 2022] DELETE /identity/v3/users/528b75d7cd774f48a612951e91d0ade3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:20.592020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:20.593588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:20.604637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:20.606390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:20.607176 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0568f-a7bd-4d97-b22d-02aa3d783d8d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:20.669287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:20.675971 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:20.676180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62097db-6080-4267-bfe4-d72905a080aa admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:20.677000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:20.677400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:20.677682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] PATH_INFO: `/v3/users/038c74a63c5b4d1da2eeaf0a905f6720` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:20.678857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038c74a63c5b4d1da2eeaf0a905f6720)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:20.680679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:20.680821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:20.681585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:20.681866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:20.682140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] PATH_INFO: `/v3/projects/2aa9f8b9f72b4753b5cd826e5bfe6a5a` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:20.683494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa9f8b9f72b4753b5cd826e5bfe6a5a)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:20.685231 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:20.705002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:20.887333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038c74a63c5b4d1da2eeaf0a905f6720'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.888316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa9f8b9f72b4753b5cd826e5bfe6a5a', 'request_id': 'req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.894312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038c74a63c5b4d1da2eeaf0a905f6720'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.895028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038c74a63c5b4d1da2eeaf0a905f6720'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.895185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa9f8b9f72b4753b5cd826e5bfe6a5a', 'request_id': 'req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.895679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038c74a63c5b4d1da2eeaf0a905f6720'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.907655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-717b35a8-d8e7-49f1-8e81-60be9ddb34f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038c74a63c5b4d1da2eeaf0a905f6720'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.980596 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa9f8b9f72b4753b5cd826e5bfe6a5a', 'request_id': 'req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.992155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] The token cache is being invalidate because project 2aa9f8b9f72b4753b5cd826e5bfe6a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:20.992660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.993773 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-eb2bcc06-f0ef-421d-b9aa-f76d47cec56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:20.995306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1265/2533] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:20 2022] DELETE /identity/v3/projects/2aa9f8b9f72b4753b5cd826e5bfe6a5a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:20.999936 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1207c-b57d-4972-964b-117531d1a53a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.000226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1207c-b57d-4972-964b-117531d1a53a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.000510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1207c-b57d-4972-964b-117531d1a53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.017906 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-87e1207c-b57d-4972-964b-117531d1a53a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:21.019170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-87e1207c-b57d-4972-964b-117531d1a53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:21.073092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1269/2534] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:20 2022] DELETE /identity/v3/users/038c74a63c5b4d1da2eeaf0a905f6720 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:21.077545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cfb3a-d599-4b36-97fa-e2554e3a90fc None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.077839 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cfb3a-d599-4b36-97fa-e2554e3a90fc None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.078117 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cfb3a-d599-4b36-97fa-e2554e3a90fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.080188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1266/2535] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:21.093802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26ce04-7421-4190-9706-9d288c739024 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.094019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26ce04-7421-4190-9706-9d288c739024 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.094295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26ce04-7421-4190-9706-9d288c739024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.096361 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4b5cfb3a-d599-4b36-97fa-e2554e3a90fc None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:21.097608 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4b5cfb3a-d599-4b36-97fa-e2554e3a90fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:21.112594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8c26ce04-7421-4190-9706-9d288c739024 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:21.113807 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8c26ce04-7421-4190-9706-9d288c739024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:21.157250 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1270/2536] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:21.167352 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cfb3a-d599-4b36-97fa-e2554e3a90fc None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:21.168679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4b5cfb3a-d599-4b36-97fa-e2554e3a90fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:21.172198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1267/2537] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:21.176270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26ce04-7421-4190-9706-9d288c739024 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:21.185432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26ce04-7421-4190-9706-9d288c739024 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:21.189986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b72bd-fd5a-438c-bccf-61362193208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:21.190601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b72bd-fd5a-438c-bccf-61362193208b admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.190881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b72bd-fd5a-438c-bccf-61362193208b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.191155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b72bd-fd5a-438c-bccf-61362193208b admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.191925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b72bd-fd5a-438c-bccf-61362193208b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:21.192261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b72bd-fd5a-438c-bccf-61362193208b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:21.193860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b72bd-fd5a-438c-bccf-61362193208b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:21.205294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1268/2538] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:16:21.209331 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b72bd-fd5a-438c-bccf-61362193208b admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:21.218521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b72bd-fd5a-438c-bccf-61362193208b admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:21.223012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075697b3-acc9-4fd6-9714-9b6d2c11e939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:21.223620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075697b3-acc9-4fd6-9714-9b6d2c11e939 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.224412 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075697b3-acc9-4fd6-9714-9b6d2c11e939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.224686 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075697b3-acc9-4fd6-9714-9b6d2c11e939 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.225485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075697b3-acc9-4fd6-9714-9b6d2c11e939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:21.227260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075697b3-acc9-4fd6-9714-9b6d2c11e939 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:21.227595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cfb3a-d599-4b36-97fa-e2554e3a90fc None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:21.232055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8095c-fbda-4b04-b07f-9b07131c0001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:21.232702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8095c-fbda-4b04-b07f-9b07131c0001 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.233017 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8095c-fbda-4b04-b07f-9b07131c0001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.233299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8095c-fbda-4b04-b07f-9b07131c0001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.243444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8095c-fbda-4b04-b07f-9b07131c0001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:21.244917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8095c-fbda-4b04-b07f-9b07131c0001 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:21.254471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1271/2539] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:16:21.312122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1269/2540] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:21 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:21.313297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8095c-fbda-4b04-b07f-9b07131c0001 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:21.322930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8095c-fbda-4b04-b07f-9b07131c0001 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:21.335497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:21.380788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4cd6eb-75a6-429b-ab0c-89917c213656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:21.381485 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cd6eb-75a6-429b-ab0c-89917c213656 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.381756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cd6eb-75a6-429b-ab0c-89917c213656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.382035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cd6eb-75a6-429b-ab0c-89917c213656 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.382742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4cd6eb-75a6-429b-ab0c-89917c213656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:21.384661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4cd6eb-75a6-429b-ab0c-89917c213656 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:21.532162 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1272/2541] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:21 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:21.533658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075697b3-acc9-4fd6-9714-9b6d2c11e939 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:21.544605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075697b3-acc9-4fd6-9714-9b6d2c11e939 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:21.550573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638eab40-dacc-49dc-b428-5c0534031474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:21.555065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638eab40-dacc-49dc-b428-5c0534031474 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.555065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638eab40-dacc-49dc-b428-5c0534031474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.555065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638eab40-dacc-49dc-b428-5c0534031474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.555065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638eab40-dacc-49dc-b428-5c0534031474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:21.555065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638eab40-dacc-49dc-b428-5c0534031474 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:21.556671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:21.567707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1270/2542] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:21.568888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4cd6eb-75a6-429b-ab0c-89917c213656 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:21.578785 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4cd6eb-75a6-429b-ab0c-89917c213656 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:21.583954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bbb5f-1d94-4915-9765-0c5fd36d6c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:21.584601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bbb5f-1d94-4915-9765-0c5fd36d6c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.584915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bbb5f-1d94-4915-9765-0c5fd36d6c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.585183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bbb5f-1d94-4915-9765-0c5fd36d6c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.585902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bbb5f-1d94-4915-9765-0c5fd36d6c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:21.587495 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bbb5f-1d94-4915-9765-0c5fd36d6c54 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:21.597764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1273/2543] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:21.602747 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638eab40-dacc-49dc-b428-5c0534031474 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:21.613246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638eab40-dacc-49dc-b428-5c0534031474 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:21.618565 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df21441-8f1b-4782-8dfc-8457e25db82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:21.619465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df21441-8f1b-4782-8dfc-8457e25db82e admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.619759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df21441-8f1b-4782-8dfc-8457e25db82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.620039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df21441-8f1b-4782-8dfc-8457e25db82e admin admin] PATH_INFO: `/v3/projects/579c7ac8cf89406f8351c307a2e9369d/users/5c3e3185330a4774b6c220090b580618/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.634985 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df21441-8f1b-4782-8dfc-8457e25db82e admin admin] RBAC: Authorizing `identity:create_grant(project_id=579c7ac8cf89406f8351c307a2e9369d, user_id=5c3e3185330a4774b6c220090b580618, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:21.639755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df21441-8f1b-4782-8dfc-8457e25db82e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:21.704822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bbb5f-1d94-4915-9765-0c5fd36d6c54 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:21.704822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e25bbb5f-1d94-4915-9765-0c5fd36d6c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:21.717605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bbb5f-1d94-4915-9765-0c5fd36d6c54 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:21.722338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:21.723198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.723482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.723768 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] PATH_INFO: `/v3/projects/029eb2a96778465bab923d2c131697e9` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.725474 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=029eb2a96778465bab923d2c131697e9)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:21.727396 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:21.742927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1271/2544] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:21 2022] PUT /identity/v3/projects/579c7ac8cf89406f8351c307a2e9369d/users/5c3e3185330a4774b6c220090b580618/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:21.754228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6eb0-0e1c-4522-a0a8-56165c4fabca None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.754595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6eb0-0e1c-4522-a0a8-56165c4fabca None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.754949 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca6eb0-0e1c-4522-a0a8-56165c4fabca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.782074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d4ca6eb0-0e1c-4522-a0a8-56165c4fabca None None] MFA Rules not processed for user `5c3e3185330a4774b6c220090b580618`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:21.783284 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d4ca6eb0-0e1c-4522-a0a8-56165c4fabca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:21.846300 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1272/2545] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:21.851241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca6eb0-0e1c-4522-a0a8-56165c4fabca None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:21.855223 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '029eb2a96778465bab923d2c131697e9', 'request_id': 'req-11b67fd4-59ac-4a7a-b018-1f8872e59128'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:21.862238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '029eb2a96778465bab923d2c131697e9', 'request_id': 'req-11b67fd4-59ac-4a7a-b018-1f8872e59128'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:21.871910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '029eb2a96778465bab923d2c131697e9', 'request_id': 'req-11b67fd4-59ac-4a7a-b018-1f8872e59128'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:21.918385 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca6eb0-0e1c-4522-a0a8-56165c4fabca None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:21.952691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] The token cache is being invalidate because project 029eb2a96778465bab923d2c131697e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:21.954161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:21.954161 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:21.955961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1274/2546] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:21 2022] DELETE /identity/v3/projects/029eb2a96778465bab923d2c131697e9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:21.960909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:21.962668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:21.976969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa5b42-7ff0-489b-8f1d-007ebab9bef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:21.977632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa5b42-7ff0-489b-8f1d-007ebab9bef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:21.977916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa5b42-7ff0-489b-8f1d-007ebab9bef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:21.978822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa5b42-7ff0-489b-8f1d-007ebab9bef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:21.979529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa5b42-7ff0-489b-8f1d-007ebab9bef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:21.984073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa5b42-7ff0-489b-8f1d-007ebab9bef6 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:21.997364 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:22.029097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b67fd4-59ac-4a7a-b018-1f8872e59128 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:22.034276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:22.035152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:22.035440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:22.035719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] PATH_INFO: `/v3/users/dcbe5d8601a34ef0bd32d0aad9c87611` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:22.036968 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcbe5d8601a34ef0bd32d0aad9c87611)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:22.038896 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:22.060243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:22.069132 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1273/2547] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:21 2022] POST /identity/v3/projects => generated 387 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:22.073138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa5b42-7ff0-489b-8f1d-007ebab9bef6 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:22.076043 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e9aa5b42-7ff0-489b-8f1d-007ebab9bef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:22.137489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa5b42-7ff0-489b-8f1d-007ebab9bef6 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:22.142029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79838f5a-d8bb-4024-a173-135efaaf7111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:22.142681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79838f5a-d8bb-4024-a173-135efaaf7111 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:22.142967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79838f5a-d8bb-4024-a173-135efaaf7111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:22.143243 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79838f5a-d8bb-4024-a173-135efaaf7111 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:22.143926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79838f5a-d8bb-4024-a173-135efaaf7111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:22.145926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79838f5a-d8bb-4024-a173-135efaaf7111 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:22.252345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcbe5d8601a34ef0bd32d0aad9c87611'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:22.260083 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcbe5d8601a34ef0bd32d0aad9c87611'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:22.264703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcbe5d8601a34ef0bd32d0aad9c87611'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:22.264703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcbe5d8601a34ef0bd32d0aad9c87611'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:22.267669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1274/2548] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:22 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:22.271957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79838f5a-d8bb-4024-a173-135efaaf7111 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:22.274580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-79838f5a-d8bb-4024-a173-135efaaf7111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:22.278674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcbe5d8601a34ef0bd32d0aad9c87611'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:22.287139 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79838f5a-d8bb-4024-a173-135efaaf7111 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:22.291715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2893e-4482-4a5c-9b73-c5d3e3d0521d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:22.292368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2893e-4482-4a5c-9b73-c5d3e3d0521d admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:22.293407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2893e-4482-4a5c-9b73-c5d3e3d0521d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:22.293407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2893e-4482-4a5c-9b73-c5d3e3d0521d admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:22.293727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2893e-4482-4a5c-9b73-c5d3e3d0521d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:22.295218 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2893e-4482-4a5c-9b73-c5d3e3d0521d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:22.305193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1275/2549] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:22.309092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2893e-4482-4a5c-9b73-c5d3e3d0521d admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:22.318268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2893e-4482-4a5c-9b73-c5d3e3d0521d admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:22.322893 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcabf4c-93ae-4d87-ad20-f3d26e35b415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:22.323715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcabf4c-93ae-4d87-ad20-f3d26e35b415 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:22.324000 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcabf4c-93ae-4d87-ad20-f3d26e35b415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:22.324277 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcabf4c-93ae-4d87-ad20-f3d26e35b415 admin admin] PATH_INFO: `/v3/projects/c3e42ba4914b457992d22f0d75625b93/users/f023232f06cb4f52a55c6acee2860cd2/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:22.338821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcabf4c-93ae-4d87-ad20-f3d26e35b415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e42ba4914b457992d22f0d75625b93, user_id=f023232f06cb4f52a55c6acee2860cd2, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:22.341462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcabf4c-93ae-4d87-ad20-f3d26e35b415 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:22.402807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1275/2550] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:21 2022] DELETE /identity/v3/users/dcbe5d8601a34ef0bd32d0aad9c87611 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:22.431688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1276/2551] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:22 2022] PUT /identity/v3/projects/c3e42ba4914b457992d22f0d75625b93/users/f023232f06cb4f52a55c6acee2860cd2/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:22.437453 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:22.500221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641461c4-eeaf-4c16-b852-5dd4d2f1dc60 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:22.555938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b781a3f-d39d-4755-b1ac-f0b9f6ca4922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:22.556677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b781a3f-d39d-4755-b1ac-f0b9f6ca4922 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:22.556996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b781a3f-d39d-4755-b1ac-f0b9f6ca4922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:22.557274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b781a3f-d39d-4755-b1ac-f0b9f6ca4922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:22.557988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b781a3f-d39d-4755-b1ac-f0b9f6ca4922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:22.559596 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b781a3f-d39d-4755-b1ac-f0b9f6ca4922 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:22.570100 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1276/2552] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:22.574301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcabf4c-93ae-4d87-ad20-f3d26e35b415 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:22.585330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcabf4c-93ae-4d87-ad20-f3d26e35b415 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:22.590033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005f9db-e9d6-4e1e-ab9c-6a6a7416fcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:22.590986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005f9db-e9d6-4e1e-ab9c-6a6a7416fcac admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:22.591264 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005f9db-e9d6-4e1e-ab9c-6a6a7416fcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:22.591596 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005f9db-e9d6-4e1e-ab9c-6a6a7416fcac admin admin] PATH_INFO: `/v3/projects/c3e42ba4914b457992d22f0d75625b93/users/f023232f06cb4f52a55c6acee2860cd2/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:22.593642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005f9db-e9d6-4e1e-ab9c-6a6a7416fcac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e42ba4914b457992d22f0d75625b93, user_id=f023232f06cb4f52a55c6acee2860cd2, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:22.596287 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005f9db-e9d6-4e1e-ab9c-6a6a7416fcac admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:22.609769 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1277/2553] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:22 2022] PUT /identity/v3/projects/c3e42ba4914b457992d22f0d75625b93/users/f023232f06cb4f52a55c6acee2860cd2/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:22.617767 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbb373-3102-418f-b758-5840e1713f9e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:22.618319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbb373-3102-418f-b758-5840e1713f9e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:22.618406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbb373-3102-418f-b758-5840e1713f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:22.650296 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e3bbb373-3102-418f-b758-5840e1713f9e None None] MFA Rules not processed for user `f023232f06cb4f52a55c6acee2860cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:22.651508 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e3bbb373-3102-418f-b758-5840e1713f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:22.683870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:22.723385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1277/2554] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:22.735204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005f9db-e9d6-4e1e-ab9c-6a6a7416fcac admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:22.736902 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5005f9db-e9d6-4e1e-ab9c-6a6a7416fcac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:22.794231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbb373-3102-418f-b758-5840e1713f9e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:22.795733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e3bbb373-3102-418f-b758-5840e1713f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:22.798680 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005f9db-e9d6-4e1e-ab9c-6a6a7416fcac admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:22.803049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2b872-73ba-4111-98da-52db12b048f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:22.803586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2b872-73ba-4111-98da-52db12b048f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:22.803869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2b872-73ba-4111-98da-52db12b048f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:22.804143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2b872-73ba-4111-98da-52db12b048f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:22.813912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2b872-73ba-4111-98da-52db12b048f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:22.815350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2b872-73ba-4111-98da-52db12b048f9 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:22.826486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1278/2555] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:22.853824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2b872-73ba-4111-98da-52db12b048f9 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:22.867269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbb373-3102-418f-b758-5840e1713f9e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:22.871840 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:22.872692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:22.873010 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:22.873287 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] PATH_INFO: `/v3/projects/ecb0cd8fe69b4b4bad42923507c1c817` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:22.874460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb0cd8fe69b4b4bad42923507c1c817)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:22.876375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:22.898919 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:22.922069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2b872-73ba-4111-98da-52db12b048f9 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:22.924226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:22.925113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:22.925394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:22.925666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] PATH_INFO: `/v3/users/5c3e3185330a4774b6c220090b580618` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:22.927439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3e3185330a4774b6c220090b580618)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:22.929453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:22.988971 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb0cd8fe69b4b4bad42923507c1c817', 'request_id': 'req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:22.996522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb0cd8fe69b4b4bad42923507c1c817', 'request_id': 'req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:23.003603 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb0cd8fe69b4b4bad42923507c1c817', 'request_id': 'req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:23.089106 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] The token cache is being invalidate because project ecb0cd8fe69b4b4bad42923507c1c817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:23.089457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:23.090029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3add98e-2f95-4d93-ae5b-40ab11cad1ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:23.091640 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1278/2556] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:22 2022] DELETE /identity/v3/projects/ecb0cd8fe69b4b4bad42923507c1c817 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:23.096379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adf1dd-87f7-40b1-876c-c325cf53c679 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.096668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adf1dd-87f7-40b1-876c-c325cf53c679 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.096976 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19adf1dd-87f7-40b1-876c-c325cf53c679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.115741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-19adf1dd-87f7-40b1-876c-c325cf53c679 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:23.117009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-19adf1dd-87f7-40b1-876c-c325cf53c679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:23.159423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3e3185330a4774b6c220090b580618'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:23.167290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3e3185330a4774b6c220090b580618'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:23.168226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3e3185330a4774b6c220090b580618'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:23.176524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:23.182437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3e3185330a4774b6c220090b580618'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:23.183432 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-48029558-f640-44c1-ae06-2856e3f80b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3e3185330a4774b6c220090b580618'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:23.193127 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1279/2557] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:23.215180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ad96a-35b7-4625-ace7-9c64d77cbe80 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.215180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ad96a-35b7-4625-ace7-9c64d77cbe80 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.215180 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ad96a-35b7-4625-ace7-9c64d77cbe80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.215953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-df1ad96a-35b7-4625-ace7-9c64d77cbe80 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:23.217189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df1ad96a-35b7-4625-ace7-9c64d77cbe80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:23.226566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1280/2558] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:23.230756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ad96a-35b7-4625-ace7-9c64d77cbe80 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:23.240884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ad96a-35b7-4625-ace7-9c64d77cbe80 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:23.245523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d5e0f-f4ec-4d66-8a58-d31cf499829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:23.246163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d5e0f-f4ec-4d66-8a58-d31cf499829a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.246446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d5e0f-f4ec-4d66-8a58-d31cf499829a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.246724 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d5e0f-f4ec-4d66-8a58-d31cf499829a admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.247446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d5e0f-f4ec-4d66-8a58-d31cf499829a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:23.247755 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d5e0f-f4ec-4d66-8a58-d31cf499829a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:23.249425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d5e0f-f4ec-4d66-8a58-d31cf499829a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:23.265061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1281/2559] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:16:23.266284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d5e0f-f4ec-4d66-8a58-d31cf499829a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:23.277350 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d5e0f-f4ec-4d66-8a58-d31cf499829a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:23.281941 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e2992-8010-47ab-880a-8b99be060162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:23.282706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e2992-8010-47ab-880a-8b99be060162 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.282990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e2992-8010-47ab-880a-8b99be060162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.283266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e2992-8010-47ab-880a-8b99be060162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.283982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e2992-8010-47ab-880a-8b99be060162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:23.285895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e2992-8010-47ab-880a-8b99be060162 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:23.299425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1279/2560] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:22 2022] DELETE /identity/v3/users/5c3e3185330a4774b6c220090b580618 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:23.303737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4354166-25af-42ca-944f-83842e83c1b5 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.304039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4354166-25af-42ca-944f-83842e83c1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.304335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4354166-25af-42ca-944f-83842e83c1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.322297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-e4354166-25af-42ca-944f-83842e83c1b5 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:23.323465 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e4354166-25af-42ca-944f-83842e83c1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:23.375019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1282/2561] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:23 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:23.378758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e2992-8010-47ab-880a-8b99be060162 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:23.388912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1280/2562] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:23.398671 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4354166-25af-42ca-944f-83842e83c1b5 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:23.400232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e4354166-25af-42ca-944f-83842e83c1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:23.448621 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e2992-8010-47ab-880a-8b99be060162 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:23.454068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3a0dd-b238-496f-94d2-65be62fea526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:23.454955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3a0dd-b238-496f-94d2-65be62fea526 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.455247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3a0dd-b238-496f-94d2-65be62fea526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.455534 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3a0dd-b238-496f-94d2-65be62fea526 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.456903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3a0dd-b238-496f-94d2-65be62fea526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:23.462221 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3a0dd-b238-496f-94d2-65be62fea526 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:23.464663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4354166-25af-42ca-944f-83842e83c1b5 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:23.478462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:23.516402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e75214-598b-4342-b711-bdd22c8cec46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:23.517092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e75214-598b-4342-b711-bdd22c8cec46 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.517378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e75214-598b-4342-b711-bdd22c8cec46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.517655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e75214-598b-4342-b711-bdd22c8cec46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.526950 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e75214-598b-4342-b711-bdd22c8cec46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:23.528378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e75214-598b-4342-b711-bdd22c8cec46 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:23.538645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1281/2563] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:16:23.549266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1283/2564] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:23 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:23.551500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e75214-598b-4342-b711-bdd22c8cec46 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:23.562467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e75214-598b-4342-b711-bdd22c8cec46 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:23.572722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c121700-9aca-4c84-9245-9735e9d54a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:23.573413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c121700-9aca-4c84-9245-9735e9d54a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.573696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c121700-9aca-4c84-9245-9735e9d54a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.573975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c121700-9aca-4c84-9245-9735e9d54a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.574716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c121700-9aca-4c84-9245-9735e9d54a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:23.576324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c121700-9aca-4c84-9245-9735e9d54a2d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:23.589889 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1282/2565] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:23.591268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3a0dd-b238-496f-94d2-65be62fea526 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:23.602005 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3a0dd-b238-496f-94d2-65be62fea526 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:23.607021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da1433-9c30-46d8-a872-d1cad1169f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:23.608622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da1433-9c30-46d8-a872-d1cad1169f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.608622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da1433-9c30-46d8-a872-d1cad1169f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.608860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da1433-9c30-46d8-a872-d1cad1169f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.609753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da1433-9c30-46d8-a872-d1cad1169f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:23.611900 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da1433-9c30-46d8-a872-d1cad1169f5a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:23.623298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1284/2566] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:23.627538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c121700-9aca-4c84-9245-9735e9d54a2d admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:23.637576 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c121700-9aca-4c84-9245-9735e9d54a2d admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:23.642348 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1887885-3ba0-4100-8802-8618bf1b8117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:23.643288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1887885-3ba0-4100-8802-8618bf1b8117 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.643643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1887885-3ba0-4100-8802-8618bf1b8117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.643991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1887885-3ba0-4100-8802-8618bf1b8117 admin admin] PATH_INFO: `/v3/projects/956372ae8e9a4420abf1f2023b3b3dcc/users/51cc5349a6ba4a20835afd81bba481ee/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.658961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1887885-3ba0-4100-8802-8618bf1b8117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=956372ae8e9a4420abf1f2023b3b3dcc, user_id=51cc5349a6ba4a20835afd81bba481ee, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:23.662335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1887885-3ba0-4100-8802-8618bf1b8117 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:23.753392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1283/2567] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:23 2022] PUT /identity/v3/projects/956372ae8e9a4420abf1f2023b3b3dcc/users/51cc5349a6ba4a20835afd81bba481ee/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:23.764859 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd7b18-1285-4207-96a3-5e712c918871 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.765155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd7b18-1285-4207-96a3-5e712c918871 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.765433 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd7b18-1285-4207-96a3-5e712c918871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.792953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-78fd7b18-1285-4207-96a3-5e712c918871 None None] MFA Rules not processed for user `51cc5349a6ba4a20835afd81bba481ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:23.794311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-78fd7b18-1285-4207-96a3-5e712c918871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:23.865436 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1285/2568] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:23.869952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1887885-3ba0-4100-8802-8618bf1b8117 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:23.935664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1887885-3ba0-4100-8802-8618bf1b8117 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:23.940450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28ea79-105d-45a9-8c6d-f923797eef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:23.941131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28ea79-105d-45a9-8c6d-f923797eef22 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:23.941417 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28ea79-105d-45a9-8c6d-f923797eef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:23.941689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28ea79-105d-45a9-8c6d-f923797eef22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:23.942575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28ea79-105d-45a9-8c6d-f923797eef22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:23.944463 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28ea79-105d-45a9-8c6d-f923797eef22 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:24.019758 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd7b18-1285-4207-96a3-5e712c918871 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:24.021870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-78fd7b18-1285-4207-96a3-5e712c918871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:24.040093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd7b18-1285-4207-96a3-5e712c918871 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:24.045624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:24.045951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1284/2569] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:23 2022] POST /identity/v3/projects => generated 389 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:24.046539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:24.046815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:24.047085 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] PATH_INFO: `/v3/projects/579c7ac8cf89406f8351c307a2e9369d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:24.048556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=579c7ac8cf89406f8351c307a2e9369d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:24.049824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28ea79-105d-45a9-8c6d-f923797eef22 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:24.050222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:24.080215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:24.138803 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28ea79-105d-45a9-8c6d-f923797eef22 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:24.144932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97831d41-e82b-4b5a-b564-69617a100973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:24.144932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97831d41-e82b-4b5a-b564-69617a100973 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:24.144932 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97831d41-e82b-4b5a-b564-69617a100973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:24.145426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97831d41-e82b-4b5a-b564-69617a100973 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:24.146780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97831d41-e82b-4b5a-b564-69617a100973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:24.149017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97831d41-e82b-4b5a-b564-69617a100973 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:24.199285 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '579c7ac8cf89406f8351c307a2e9369d', 'request_id': 'req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:24.205096 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '579c7ac8cf89406f8351c307a2e9369d', 'request_id': 'req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:24.211560 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '579c7ac8cf89406f8351c307a2e9369d', 'request_id': 'req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:24.267004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1285/2570] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:24 2022] POST /identity/v3/users => generated 301 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:24.270837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97831d41-e82b-4b5a-b564-69617a100973 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:24.280643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97831d41-e82b-4b5a-b564-69617a100973 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:24.291177 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] The token cache is being invalidate because project 579c7ac8cf89406f8351c307a2e9369d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:24.291381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:24.292334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:24.293961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1286/2571] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:24 2022] DELETE /identity/v3/projects/579c7ac8cf89406f8351c307a2e9369d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:24.297839 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:24.300205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:24.339517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d5c84-eb7a-4fa9-ac6a-7980207a2c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:24.340191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d5c84-eb7a-4fa9-ac6a-7980207a2c6f admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:24.340479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d5c84-eb7a-4fa9-ac6a-7980207a2c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:24.340783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d5c84-eb7a-4fa9-ac6a-7980207a2c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:24.341616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d5c84-eb7a-4fa9-ac6a-7980207a2c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:24.343221 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d5c84-eb7a-4fa9-ac6a-7980207a2c6f admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:24.353742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1286/2572] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:24.357619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d5c84-eb7a-4fa9-ac6a-7980207a2c6f admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:24.359249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f09d5c84-eb7a-4fa9-ac6a-7980207a2c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:24.367253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adabe65-f2b7-4ffb-bc3a-d2212e7f29e1 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:24.372055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d5c84-eb7a-4fa9-ac6a-7980207a2c6f admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:24.376215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:24.376215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:24.376215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:24.376215 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] PATH_INFO: `/v3/users/f023232f06cb4f52a55c6acee2860cd2` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:24.376661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f023232f06cb4f52a55c6acee2860cd2)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:24.376784 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63011f7f-d053-442d-8914-be8f521d4da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:24.376911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:24.377426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63011f7f-d053-442d-8914-be8f521d4da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:24.377709 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63011f7f-d053-442d-8914-be8f521d4da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:24.377984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63011f7f-d053-442d-8914-be8f521d4da7 admin admin] PATH_INFO: `/v3/projects/a0093767b180470f8cde8e1b59eae5bb/users/d0fda23415a3475cbe3486601256f681/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:24.394571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63011f7f-d053-442d-8914-be8f521d4da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0093767b180470f8cde8e1b59eae5bb, user_id=d0fda23415a3475cbe3486601256f681, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:24.398716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63011f7f-d053-442d-8914-be8f521d4da7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:24.449566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:24.478937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1287/2573] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:24 2022] PUT /identity/v3/projects/a0093767b180470f8cde8e1b59eae5bb/users/d0fda23415a3475cbe3486601256f681/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:24.482857 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63011f7f-d053-442d-8914-be8f521d4da7 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:24.538154 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f023232f06cb4f52a55c6acee2860cd2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:24.543047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63011f7f-d053-442d-8914-be8f521d4da7 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:24.545165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f023232f06cb4f52a55c6acee2860cd2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:24.545873 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f023232f06cb4f52a55c6acee2860cd2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:24.546494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f023232f06cb4f52a55c6acee2860cd2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:24.548652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34aaff3-6c26-41a2-9c17-0e4b17be6aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:24.549333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34aaff3-6c26-41a2-9c17-0e4b17be6aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:24.549572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34aaff3-6c26-41a2-9c17-0e4b17be6aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:24.549662 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34aaff3-6c26-41a2-9c17-0e4b17be6aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:24.550375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34aaff3-6c26-41a2-9c17-0e4b17be6aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:24.551990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34aaff3-6c26-41a2-9c17-0e4b17be6aa1 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:24.562060 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1288/2574] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:24.563679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f023232f06cb4f52a55c6acee2860cd2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:24.566100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34aaff3-6c26-41a2-9c17-0e4b17be6aa1 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:24.567860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c34aaff3-6c26-41a2-9c17-0e4b17be6aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:24.580106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34aaff3-6c26-41a2-9c17-0e4b17be6aa1 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:24.585171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb805eb8-243d-4d18-ae46-11c5f889393e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:24.586036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb805eb8-243d-4d18-ae46-11c5f889393e admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:24.586319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb805eb8-243d-4d18-ae46-11c5f889393e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:24.586594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb805eb8-243d-4d18-ae46-11c5f889393e admin admin] PATH_INFO: `/v3/projects/a0093767b180470f8cde8e1b59eae5bb/users/d0fda23415a3475cbe3486601256f681/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:24.588510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb805eb8-243d-4d18-ae46-11c5f889393e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0093767b180470f8cde8e1b59eae5bb, user_id=d0fda23415a3475cbe3486601256f681, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:24.591170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb805eb8-243d-4d18-ae46-11c5f889393e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:24.601796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1289/2575] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:24 2022] PUT /identity/v3/projects/a0093767b180470f8cde8e1b59eae5bb/users/d0fda23415a3475cbe3486601256f681/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:24.611985 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a901f0-f86e-4637-b335-1c6e234d3a1f None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:24.612275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a901f0-f86e-4637-b335-1c6e234d3a1f None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:24.612552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a901f0-f86e-4637-b335-1c6e234d3a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:24.639311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f8a901f0-f86e-4637-b335-1c6e234d3a1f None None] MFA Rules not processed for user `d0fda23415a3475cbe3486601256f681`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:24.640813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f8a901f0-f86e-4637-b335-1c6e234d3a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:24.662836 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1287/2576] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:24 2022] DELETE /identity/v3/users/f023232f06cb4f52a55c6acee2860cd2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:24.712604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1290/2577] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:24.723397 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:24.725430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:24.758441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:24.784875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2228a-09bd-4858-9d78-86907d451755 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:24.789143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119e0cd-b2c0-4c67-bd18-604c9d635266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:24.789775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119e0cd-b2c0-4c67-bd18-604c9d635266 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:24.790064 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119e0cd-b2c0-4c67-bd18-604c9d635266 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:24.790347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119e0cd-b2c0-4c67-bd18-604c9d635266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:24.832043 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:24.852595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119e0cd-b2c0-4c67-bd18-604c9d635266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:24.854082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119e0cd-b2c0-4c67-bd18-604c9d635266 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:24.864262 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1288/2578] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:25.002679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a901f0-f86e-4637-b335-1c6e234d3a1f None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:25.069164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a901f0-f86e-4637-b335-1c6e234d3a1f None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:25.084885 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119e0cd-b2c0-4c67-bd18-604c9d635266 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:25.086370 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3119e0cd-b2c0-4c67-bd18-604c9d635266 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:25.137392 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:25.139637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:25.139927 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:25.140218 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] PATH_INFO: `/v3/users/51cc5349a6ba4a20835afd81bba481ee` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:25.141505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51cc5349a6ba4a20835afd81bba481ee)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:25.143451 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:25.185686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119e0cd-b2c0-4c67-bd18-604c9d635266 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:25.190220 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:25.191075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:25.191356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:25.191629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] PATH_INFO: `/v3/projects/c3e42ba4914b457992d22f0d75625b93` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:25.192865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e42ba4914b457992d22f0d75625b93)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:25.194789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:25.374510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51cc5349a6ba4a20835afd81bba481ee'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:25.375875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e42ba4914b457992d22f0d75625b93', 'request_id': 'req-281d592f-df18-4c2e-bd74-db4c52b2ec09'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:25.382336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e42ba4914b457992d22f0d75625b93', 'request_id': 'req-281d592f-df18-4c2e-bd74-db4c52b2ec09'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:25.383319 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51cc5349a6ba4a20835afd81bba481ee'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:25.384074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51cc5349a6ba4a20835afd81bba481ee'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:25.388717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e42ba4914b457992d22f0d75625b93', 'request_id': 'req-281d592f-df18-4c2e-bd74-db4c52b2ec09'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:25.397769 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51cc5349a6ba4a20835afd81bba481ee'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:25.398502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ca2767ea-3db4-4133-b236-6e7b5a663ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51cc5349a6ba4a20835afd81bba481ee'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:25.473294 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] The token cache is being invalidate because project c3e42ba4914b457992d22f0d75625b93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:25.474395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:25.475175 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-281d592f-df18-4c2e-bd74-db4c52b2ec09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:25.477054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1289/2579] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:25 2022] DELETE /identity/v3/projects/c3e42ba4914b457992d22f0d75625b93 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:25.482012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34d5b0-b71d-490d-a45b-51df7df873bd None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:25.482334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34d5b0-b71d-490d-a45b-51df7df873bd None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:25.482578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34d5b0-b71d-490d-a45b-51df7df873bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:25.496171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:25.502157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-4f34d5b0-b71d-490d-a45b-51df7df873bd None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:25.503386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4f34d5b0-b71d-490d-a45b-51df7df873bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:25.524449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1291/2580] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:25 2022] DELETE /identity/v3/users/51cc5349a6ba4a20835afd81bba481ee => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:25.529013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf8340-467b-4fae-b648-48aecf42e61e None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:25.529462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf8340-467b-4fae-b648-48aecf42e61e None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:25.530341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf8340-467b-4fae-b648-48aecf42e61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:25.536951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:25.547771 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-3faf8340-467b-4fae-b648-48aecf42e61e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:25.549135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3faf8340-467b-4fae-b648-48aecf42e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:25.569455 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1290/2581] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:25.576923 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7296-edd5-4932-9ea8-82203a72acf1 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:25.577303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7296-edd5-4932-9ea8-82203a72acf1 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:25.577678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b7296-edd5-4932-9ea8-82203a72acf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:25.596532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-096b7296-edd5-4932-9ea8-82203a72acf1 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:25.597801 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-096b7296-edd5-4932-9ea8-82203a72acf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:25.617091 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1292/2582] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:25.626575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf8340-467b-4fae-b648-48aecf42e61e None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:25.627952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3faf8340-467b-4fae-b648-48aecf42e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:25.658818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1291/2583] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:25.663629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b7296-edd5-4932-9ea8-82203a72acf1 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:25.673466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b7296-edd5-4932-9ea8-82203a72acf1 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:25.681118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e749ef-8499-4035-b622-7de81a10d415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:25.683322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e749ef-8499-4035-b622-7de81a10d415 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:25.683322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e749ef-8499-4035-b622-7de81a10d415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:25.683322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e749ef-8499-4035-b622-7de81a10d415 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:25.683322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e749ef-8499-4035-b622-7de81a10d415 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:25.683651 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e749ef-8499-4035-b622-7de81a10d415 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:25.685536 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e749ef-8499-4035-b622-7de81a10d415 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:25.689851 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf8340-467b-4fae-b648-48aecf42e61e None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:25.693639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034f911-2856-4bba-96dd-5846f1184b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:25.694320 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034f911-2856-4bba-96dd-5846f1184b0b service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:25.694608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034f911-2856-4bba-96dd-5846f1184b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:25.694882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034f911-2856-4bba-96dd-5846f1184b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:25.698532 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1292/2584] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:16:25.702710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e749ef-8499-4035-b622-7de81a10d415 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:25.709491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034f911-2856-4bba-96dd-5846f1184b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:25.712224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034f911-2856-4bba-96dd-5846f1184b0b service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:25.713054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e749ef-8499-4035-b622-7de81a10d415 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:25.718439 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac235f-eaac-4e48-85f8-33c977efa225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:25.719124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac235f-eaac-4e48-85f8-33c977efa225 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:25.719403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac235f-eaac-4e48-85f8-33c977efa225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:25.719675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac235f-eaac-4e48-85f8-33c977efa225 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:25.720407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac235f-eaac-4e48-85f8-33c977efa225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:25.722412 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac235f-eaac-4e48-85f8-33c977efa225 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:25.723116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1293/2585] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:25.897488 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1293/2586] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:25 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:25.901369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034f911-2856-4bba-96dd-5846f1184b0b service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:25.911002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034f911-2856-4bba-96dd-5846f1184b0b service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:25.969259 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f2cbe-4ff3-48bd-bf05-46747c601567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:25.970031 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f2cbe-4ff3-48bd-bf05-46747c601567 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:25.970316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f2cbe-4ff3-48bd-bf05-46747c601567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:25.970594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f2cbe-4ff3-48bd-bf05-46747c601567 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:25.971404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f2cbe-4ff3-48bd-bf05-46747c601567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:25.973463 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f2cbe-4ff3-48bd-bf05-46747c601567 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:26.061365 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1294/2587] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:25 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:26.065493 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac235f-eaac-4e48-85f8-33c977efa225 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:26.076284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac235f-eaac-4e48-85f8-33c977efa225 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:26.081737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04ee66-5abf-4124-b48f-12509f73e730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:26.082557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04ee66-5abf-4124-b48f-12509f73e730 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:26.083422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04ee66-5abf-4124-b48f-12509f73e730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:26.083802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04ee66-5abf-4124-b48f-12509f73e730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:26.084635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04ee66-5abf-4124-b48f-12509f73e730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:26.086478 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04ee66-5abf-4124-b48f-12509f73e730 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:26.103424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1294/2588] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:26.107742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f2cbe-4ff3-48bd-bf05-46747c601567 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:26.118516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f2cbe-4ff3-48bd-bf05-46747c601567 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:26.123033 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f2efb-8e51-4e68-858c-d4324bc0e9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:26.123683 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f2efb-8e51-4e68-858c-d4324bc0e9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:26.123964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f2efb-8e51-4e68-858c-d4324bc0e9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:26.124241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f2efb-8e51-4e68-858c-d4324bc0e9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:26.125136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f2efb-8e51-4e68-858c-d4324bc0e9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:26.126708 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f2efb-8e51-4e68-858c-d4324bc0e9a8 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:26.137143 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1295/2589] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:26.141228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04ee66-5abf-4124-b48f-12509f73e730 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:26.151359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04ee66-5abf-4124-b48f-12509f73e730 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:26.155890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef4813-a5da-49d8-8d85-42eda91cede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:26.156771 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef4813-a5da-49d8-8d85-42eda91cede4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:26.157199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef4813-a5da-49d8-8d85-42eda91cede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:26.157501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef4813-a5da-49d8-8d85-42eda91cede4 admin admin] PATH_INFO: `/v3/projects/66ac7c8ec8bc4299a262f166e9481602/users/b7ed9ddfdd6842619789e5adcaa1d6ea/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:26.171909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef4813-a5da-49d8-8d85-42eda91cede4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ac7c8ec8bc4299a262f166e9481602, user_id=b7ed9ddfdd6842619789e5adcaa1d6ea, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:26.174649 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef4813-a5da-49d8-8d85-42eda91cede4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:26.194625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f2efb-8e51-4e68-858c-d4324bc0e9a8 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:26.196341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-540f2efb-8e51-4e68-858c-d4324bc0e9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:26.220812 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f2efb-8e51-4e68-858c-d4324bc0e9a8 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:26.227312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:26.228246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:26.228600 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:26.228971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] PATH_INFO: `/v3/projects/956372ae8e9a4420abf1f2023b3b3dcc` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:26.230241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=956372ae8e9a4420abf1f2023b3b3dcc)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:26.232178 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:26.258425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:26.281557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1295/2590] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:26 2022] PUT /identity/v3/projects/66ac7c8ec8bc4299a262f166e9481602/users/b7ed9ddfdd6842619789e5adcaa1d6ea/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:26.299038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f6b70-a04a-400b-8916-9c96520852cf None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:26.299038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f6b70-a04a-400b-8916-9c96520852cf None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:26.299683 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f6b70-a04a-400b-8916-9c96520852cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:26.329193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-dd7f6b70-a04a-400b-8916-9c96520852cf None None] MFA Rules not processed for user `b7ed9ddfdd6842619789e5adcaa1d6ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:26.330533 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-dd7f6b70-a04a-400b-8916-9c96520852cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:26.365827 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:26.387667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '956372ae8e9a4420abf1f2023b3b3dcc', 'request_id': 'req-90403674-1d79-4347-ac62-9dcfaebec261'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:26.397583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '956372ae8e9a4420abf1f2023b3b3dcc', 'request_id': 'req-90403674-1d79-4347-ac62-9dcfaebec261'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:26.412666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1296/2591] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:26.416855 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f6b70-a04a-400b-8916-9c96520852cf None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:26.484049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '956372ae8e9a4420abf1f2023b3b3dcc', 'request_id': 'req-90403674-1d79-4347-ac62-9dcfaebec261'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:26.487547 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f6b70-a04a-400b-8916-9c96520852cf None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:26.503428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] The token cache is being invalidate because project 956372ae8e9a4420abf1f2023b3b3dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:26.503925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:26.504941 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:26.510729 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1296/2592] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:26 2022] DELETE /identity/v3/projects/956372ae8e9a4420abf1f2023b3b3dcc => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:26.514828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:26.516263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:26.570325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb264d-92ed-4cff-8ffb-96ed37dd4247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:26.570325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb264d-92ed-4cff-8ffb-96ed37dd4247 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:26.570325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb264d-92ed-4cff-8ffb-96ed37dd4247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:26.570325 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb264d-92ed-4cff-8ffb-96ed37dd4247 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:26.570914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb264d-92ed-4cff-8ffb-96ed37dd4247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:26.572866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb264d-92ed-4cff-8ffb-96ed37dd4247 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:26.586525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90403674-1d79-4347-ac62-9dcfaebec261 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:26.591519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:26.592384 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:26.592724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:26.593058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] PATH_INFO: `/v3/users/d0fda23415a3475cbe3486601256f681` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:26.594339 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0fda23415a3475cbe3486601256f681)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:26.596309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:26.646237 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1297/2593] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:26 2022] POST /identity/v3/projects => generated 379 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:26.650217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb264d-92ed-4cff-8ffb-96ed37dd4247 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:26.651955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-2bfb264d-92ed-4cff-8ffb-96ed37dd4247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:26.717635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb264d-92ed-4cff-8ffb-96ed37dd4247 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:26.722594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87144c2-a10e-4648-bfb8-83767d3356b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:26.723282 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87144c2-a10e-4648-bfb8-83767d3356b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:26.723622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87144c2-a10e-4648-bfb8-83767d3356b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:26.724488 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87144c2-a10e-4648-bfb8-83767d3356b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:26.725216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87144c2-a10e-4648-bfb8-83767d3356b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:26.727348 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87144c2-a10e-4648-bfb8-83767d3356b0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:26.734574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:26.796096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0fda23415a3475cbe3486601256f681'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:26.803743 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0fda23415a3475cbe3486601256f681'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:26.804411 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0fda23415a3475cbe3486601256f681'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:26.816749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0fda23415a3475cbe3486601256f681'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:26.817778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0fda23415a3475cbe3486601256f681'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:26.827497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1298/2594] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:26 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:26.831241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87144c2-a10e-4648-bfb8-83767d3356b0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:26.833531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b87144c2-a10e-4648-bfb8-83767d3356b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:26.845440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87144c2-a10e-4648-bfb8-83767d3356b0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:26.850849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f5f07-a70a-415a-af60-ddcf814aabdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:26.852000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f5f07-a70a-415a-af60-ddcf814aabdc admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:26.854403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f5f07-a70a-415a-af60-ddcf814aabdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:26.854403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f5f07-a70a-415a-af60-ddcf814aabdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:26.854403 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f5f07-a70a-415a-af60-ddcf814aabdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:26.854972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f5f07-a70a-415a-af60-ddcf814aabdc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:26.869253 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1299/2595] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:26.870224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f5f07-a70a-415a-af60-ddcf814aabdc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:26.880538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f5f07-a70a-415a-af60-ddcf814aabdc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:26.885299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2040726-8712-40d5-b83b-a9b2561b6333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:26.886165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040726-8712-40d5-b83b-a9b2561b6333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:26.886446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040726-8712-40d5-b83b-a9b2561b6333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:26.886726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040726-8712-40d5-b83b-a9b2561b6333 admin admin] PATH_INFO: `/v3/projects/eda08082be72494eb3d90ee8e69388d5/users/70fd765d1f1e477481cc89c1b2039005/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:26.902613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2040726-8712-40d5-b83b-a9b2561b6333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda08082be72494eb3d90ee8e69388d5, user_id=70fd765d1f1e477481cc89c1b2039005, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:26.905380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2040726-8712-40d5-b83b-a9b2561b6333 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:26.924065 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1297/2596] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:26 2022] DELETE /identity/v3/users/d0fda23415a3475cbe3486601256f681 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:26.958930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1300/2597] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:26 2022] PUT /identity/v3/projects/eda08082be72494eb3d90ee8e69388d5/users/70fd765d1f1e477481cc89c1b2039005/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:26.963796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:26.995705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:27.029106 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950acad-6203-49af-8fd8-c15df9fef018 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:27.096038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22e14b-f3a6-40b9-b940-20f49983171a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:27.096884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22e14b-f3a6-40b9-b940-20f49983171a admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:27.097606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22e14b-f3a6-40b9-b940-20f49983171a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:27.097606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22e14b-f3a6-40b9-b940-20f49983171a admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:27.098309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22e14b-f3a6-40b9-b940-20f49983171a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:27.100536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22e14b-f3a6-40b9-b940-20f49983171a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:27.115123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1298/2598] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:27.117720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2040726-8712-40d5-b83b-a9b2561b6333 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:27.128593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2040726-8712-40d5-b83b-a9b2561b6333 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:27.134145 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01259705-2ee3-4f5c-ac91-6d835523a4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:27.135138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01259705-2ee3-4f5c-ac91-6d835523a4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:27.135409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01259705-2ee3-4f5c-ac91-6d835523a4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:27.135740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01259705-2ee3-4f5c-ac91-6d835523a4b2 admin admin] PATH_INFO: `/v3/projects/eda08082be72494eb3d90ee8e69388d5/users/70fd765d1f1e477481cc89c1b2039005/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:27.138700 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01259705-2ee3-4f5c-ac91-6d835523a4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda08082be72494eb3d90ee8e69388d5, user_id=70fd765d1f1e477481cc89c1b2039005, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:27.144045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01259705-2ee3-4f5c-ac91-6d835523a4b2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:27.163187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1301/2599] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:27 2022] PUT /identity/v3/projects/eda08082be72494eb3d90ee8e69388d5/users/70fd765d1f1e477481cc89c1b2039005/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:27.174102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e8aff-c6bc-4309-90f6-c789a2ea62b2 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:27.174724 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e8aff-c6bc-4309-90f6-c789a2ea62b2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:27.176442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e8aff-c6bc-4309-90f6-c789a2ea62b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:27.207421 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d63e8aff-c6bc-4309-90f6-c789a2ea62b2 None None] MFA Rules not processed for user `70fd765d1f1e477481cc89c1b2039005`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:27.209250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d63e8aff-c6bc-4309-90f6-c789a2ea62b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:27.316080 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1299/2600] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:27.329872 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01259705-2ee3-4f5c-ac91-6d835523a4b2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:27.331744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-01259705-2ee3-4f5c-ac91-6d835523a4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:27.398677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01259705-2ee3-4f5c-ac91-6d835523a4b2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:27.402582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beddd1-9f48-477c-8709-3dcb0770d827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:27.403757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beddd1-9f48-477c-8709-3dcb0770d827 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:27.404053 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beddd1-9f48-477c-8709-3dcb0770d827 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:27.404453 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beddd1-9f48-477c-8709-3dcb0770d827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:27.414146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e8aff-c6bc-4309-90f6-c789a2ea62b2 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:27.415541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d63e8aff-c6bc-4309-90f6-c789a2ea62b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:27.415788 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beddd1-9f48-477c-8709-3dcb0770d827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:27.418693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beddd1-9f48-477c-8709-3dcb0770d827 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:27.430814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1302/2601] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:27.538104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:27.543118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e8aff-c6bc-4309-90f6-c789a2ea62b2 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:27.550915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:27.551888 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:27.558544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:27.558544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] PATH_INFO: `/v3/projects/a0093767b180470f8cde8e1b59eae5bb` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:27.559208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0093767b180470f8cde8e1b59eae5bb)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:27.561276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:27.568406 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beddd1-9f48-477c-8709-3dcb0770d827 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:27.570654 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-89beddd1-9f48-477c-8709-3dcb0770d827 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:27.585650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beddd1-9f48-477c-8709-3dcb0770d827 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:27.591823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d8dc4-a0d6-49de-ab24-d5f333c06a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:27.595681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d8dc4-a0d6-49de-ab24-d5f333c06a43 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:27.595681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d8dc4-a0d6-49de-ab24-d5f333c06a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:27.595681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d8dc4-a0d6-49de-ab24-d5f333c06a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:27.595681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d08d8dc4-a0d6-49de-ab24-d5f333c06a43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:27.666668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d8dc4-a0d6-49de-ab24-d5f333c06a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:27.669107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d8dc4-a0d6-49de-ab24-d5f333c06a43 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:27.683398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1303/2602] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:27.717923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0093767b180470f8cde8e1b59eae5bb', 'request_id': 'req-df8d5d94-d840-4264-93c4-c2fd843e497b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:27.725539 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0093767b180470f8cde8e1b59eae5bb', 'request_id': 'req-df8d5d94-d840-4264-93c4-c2fd843e497b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:27.842320 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0093767b180470f8cde8e1b59eae5bb', 'request_id': 'req-df8d5d94-d840-4264-93c4-c2fd843e497b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:27.854332 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] The token cache is being invalidate because project a0093767b180470f8cde8e1b59eae5bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:27.854618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:27.855229 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-df8d5d94-d840-4264-93c4-c2fd843e497b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:27.856758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1300/2603] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:27 2022] DELETE /identity/v3/projects/a0093767b180470f8cde8e1b59eae5bb => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:27.862196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e4bbb-f900-44e2-bc15-0a468715d1bb None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:27.862649 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e4bbb-f900-44e2-bc15-0a468715d1bb None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:27.862761 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e4bbb-f900-44e2-bc15-0a468715d1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:27.881459 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-799e4bbb-f900-44e2-bc15-0a468715d1bb None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:27.882635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-799e4bbb-f900-44e2-bc15-0a468715d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:27.921582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:27.942891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1304/2604] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:27.950934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb131c-65a1-4489-a56e-3056913f997c None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:27.951247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb131c-65a1-4489-a56e-3056913f997c None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:27.951524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb131c-65a1-4489-a56e-3056913f997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:27.973149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-72fb131c-65a1-4489-a56e-3056913f997c None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:27.974870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-72fb131c-65a1-4489-a56e-3056913f997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:27.982956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1301/2605] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:27.987917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e4bbb-f900-44e2-bc15-0a468715d1bb None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:27.998229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e4bbb-f900-44e2-bc15-0a468715d1bb None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:28.003080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615ad06-8058-4be1-ae73-ec869ecb1ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:28.003747 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615ad06-8058-4be1-ae73-ec869ecb1ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:28.004026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615ad06-8058-4be1-ae73-ec869ecb1ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:28.004299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615ad06-8058-4be1-ae73-ec869ecb1ff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:28.005130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615ad06-8058-4be1-ae73-ec869ecb1ff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:28.005617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615ad06-8058-4be1-ae73-ec869ecb1ff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:28.007039 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615ad06-8058-4be1-ae73-ec869ecb1ff9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:28.019243 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1305/2606] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:16:28.028677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb131c-65a1-4489-a56e-3056913f997c None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:28.039753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb131c-65a1-4489-a56e-3056913f997c None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:28.044984 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb6cd9-56ba-4342-88ab-f6c2c13d08e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:28.045915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb6cd9-56ba-4342-88ab-f6c2c13d08e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:28.046269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb6cd9-56ba-4342-88ab-f6c2c13d08e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:28.046612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb6cd9-56ba-4342-88ab-f6c2c13d08e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:28.047398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb6cd9-56ba-4342-88ab-f6c2c13d08e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:28.049488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb6cd9-56ba-4342-88ab-f6c2c13d08e2 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:28.206286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1302/2607] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:28 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:28.210263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615ad06-8058-4be1-ae73-ec869ecb1ff9 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:28.220553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615ad06-8058-4be1-ae73-ec869ecb1ff9 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:28.279379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b96e57-e6f7-4802-a762-4a385e5d05c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:28.280051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b96e57-e6f7-4802-a762-4a385e5d05c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:28.280333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b96e57-e6f7-4802-a762-4a385e5d05c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:28.281543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b96e57-e6f7-4802-a762-4a385e5d05c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:28.281543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b96e57-e6f7-4802-a762-4a385e5d05c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:28.283531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b96e57-e6f7-4802-a762-4a385e5d05c4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:28.403960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1306/2608] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:28 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:28.408200 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb6cd9-56ba-4342-88ab-f6c2c13d08e2 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:28.420801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb6cd9-56ba-4342-88ab-f6c2c13d08e2 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:28.427535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238deaa-9fb4-4b32-a615-4941a954a147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:28.428030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238deaa-9fb4-4b32-a615-4941a954a147 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:28.428316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238deaa-9fb4-4b32-a615-4941a954a147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:28.428641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238deaa-9fb4-4b32-a615-4941a954a147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:28.429585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238deaa-9fb4-4b32-a615-4941a954a147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:28.431269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238deaa-9fb4-4b32-a615-4941a954a147 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:28.444535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1303/2609] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:28.448675 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b96e57-e6f7-4802-a762-4a385e5d05c4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:28.459303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b96e57-e6f7-4802-a762-4a385e5d05c4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:28.465898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c215e-57de-4077-a203-1adfa12cd058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:28.466609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c215e-57de-4077-a203-1adfa12cd058 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:28.466912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c215e-57de-4077-a203-1adfa12cd058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:28.467218 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c215e-57de-4077-a203-1adfa12cd058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:28.467963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c215e-57de-4077-a203-1adfa12cd058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:28.469674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c215e-57de-4077-a203-1adfa12cd058 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:28.489079 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1307/2610] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:28.494652 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238deaa-9fb4-4b32-a615-4941a954a147 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:28.514721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238deaa-9fb4-4b32-a615-4941a954a147 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:28.519910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fe8d9-c177-4975-a9a2-193eee994dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:28.521664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe8d9-c177-4975-a9a2-193eee994dad admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:28.522012 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe8d9-c177-4975-a9a2-193eee994dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:28.522354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe8d9-c177-4975-a9a2-193eee994dad admin admin] PATH_INFO: `/v3/projects/1126dcf7b2a6440bba4c059edd4320ed/users/90cae4f03afd46e4b4bfcb93aec4cd1c/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:28.548593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fe8d9-c177-4975-a9a2-193eee994dad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1126dcf7b2a6440bba4c059edd4320ed, user_id=90cae4f03afd46e4b4bfcb93aec4cd1c, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:28.551450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fe8d9-c177-4975-a9a2-193eee994dad admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:28.661656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1304/2611] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:28 2022] PUT /identity/v3/projects/1126dcf7b2a6440bba4c059edd4320ed/users/90cae4f03afd46e4b4bfcb93aec4cd1c/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:28.680460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e19f1-12de-4b6f-b4cc-93ebbbdb4591 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:28.680460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e19f1-12de-4b6f-b4cc-93ebbbdb4591 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:28.680460 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e19f1-12de-4b6f-b4cc-93ebbbdb4591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:28.707008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-128e19f1-12de-4b6f-b4cc-93ebbbdb4591 None None] MFA Rules not processed for user `90cae4f03afd46e4b4bfcb93aec4cd1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:28.709356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-128e19f1-12de-4b6f-b4cc-93ebbbdb4591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:28.791420 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1308/2612] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:28.795491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fe8d9-c177-4975-a9a2-193eee994dad admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:28.858926 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fe8d9-c177-4975-a9a2-193eee994dad admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:28.864055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705b14-b7e9-4542-84e2-a6269f0389d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:28.865357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705b14-b7e9-4542-84e2-a6269f0389d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:28.865720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705b14-b7e9-4542-84e2-a6269f0389d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:28.866063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705b14-b7e9-4542-84e2-a6269f0389d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:28.866824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705b14-b7e9-4542-84e2-a6269f0389d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:28.868811 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705b14-b7e9-4542-84e2-a6269f0389d5 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:28.966740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1305/2613] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:28 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:28.970574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e19f1-12de-4b6f-b4cc-93ebbbdb4591 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:28.983284 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e19f1-12de-4b6f-b4cc-93ebbbdb4591 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:29.017418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:29.041972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e0d0c-6f37-48ab-b582-1c0879056eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:29.042669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e0d0c-6f37-48ab-b582-1c0879056eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:29.042953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e0d0c-6f37-48ab-b582-1c0879056eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:29.043238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e0d0c-6f37-48ab-b582-1c0879056eb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:29.043943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e0d0c-6f37-48ab-b582-1c0879056eb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:29.045957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e0d0c-6f37-48ab-b582-1c0879056eb4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:29.191255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1309/2614] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:28 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:29.195879 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705b14-b7e9-4542-84e2-a6269f0389d5 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:29.206164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705b14-b7e9-4542-84e2-a6269f0389d5 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:29.211382 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35ce97-7736-4d09-ab55-986b9257a120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:29.212073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35ce97-7736-4d09-ab55-986b9257a120 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:29.212407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35ce97-7736-4d09-ab55-986b9257a120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:29.212632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35ce97-7736-4d09-ab55-986b9257a120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:29.213395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35ce97-7736-4d09-ab55-986b9257a120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:29.214973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35ce97-7736-4d09-ab55-986b9257a120 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:29.217170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:29.226806 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1306/2615] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:29.230981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e0d0c-6f37-48ab-b582-1c0879056eb4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:29.241269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e0d0c-6f37-48ab-b582-1c0879056eb4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:29.245898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95392c-5f84-4daf-b89e-d4aff9f8a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:29.246773 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95392c-5f84-4daf-b89e-d4aff9f8a6da admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:29.247060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95392c-5f84-4daf-b89e-d4aff9f8a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:29.247342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95392c-5f84-4daf-b89e-d4aff9f8a6da admin admin] PATH_INFO: `/v3/projects/dec9bc42b292433db35c5eb04e7ee7f8/users/455836cdf93740ecb13f33cb6cacc53b/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:29.262941 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95392c-5f84-4daf-b89e-d4aff9f8a6da admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec9bc42b292433db35c5eb04e7ee7f8, user_id=455836cdf93740ecb13f33cb6cacc53b, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:29.265633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95392c-5f84-4daf-b89e-d4aff9f8a6da admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:29.368273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1310/2616] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:29 2022] PUT /identity/v3/projects/dec9bc42b292433db35c5eb04e7ee7f8/users/455836cdf93740ecb13f33cb6cacc53b/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:29.372172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35ce97-7736-4d09-ab55-986b9257a120 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:29.382627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35ce97-7736-4d09-ab55-986b9257a120 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:29.438018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81281399-08ed-4900-9087-972aa76b4062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:29.438677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81281399-08ed-4900-9087-972aa76b4062 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:29.438968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81281399-08ed-4900-9087-972aa76b4062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:29.439250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81281399-08ed-4900-9087-972aa76b4062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:29.439979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81281399-08ed-4900-9087-972aa76b4062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:29.441607 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81281399-08ed-4900-9087-972aa76b4062 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:29.452373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1307/2617] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:29.456443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95392c-5f84-4daf-b89e-d4aff9f8a6da admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:29.467017 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95392c-5f84-4daf-b89e-d4aff9f8a6da admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:29.471480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054e39e-743e-4a09-be38-64ef3369aad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:29.472340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e39e-743e-4a09-be38-64ef3369aad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:29.472619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e39e-743e-4a09-be38-64ef3369aad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:29.472939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e39e-743e-4a09-be38-64ef3369aad0 admin admin] PATH_INFO: `/v3/projects/dec9bc42b292433db35c5eb04e7ee7f8/users/455836cdf93740ecb13f33cb6cacc53b/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:29.474857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054e39e-743e-4a09-be38-64ef3369aad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec9bc42b292433db35c5eb04e7ee7f8, user_id=455836cdf93740ecb13f33cb6cacc53b, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:29.477501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054e39e-743e-4a09-be38-64ef3369aad0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:29.487770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1311/2618] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:29 2022] PUT /identity/v3/projects/dec9bc42b292433db35c5eb04e7ee7f8/users/455836cdf93740ecb13f33cb6cacc53b/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:29.497605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1acf14-6eb3-43ab-8bb8-cf02845f95c2 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:29.497901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1acf14-6eb3-43ab-8bb8-cf02845f95c2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:29.498182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1acf14-6eb3-43ab-8bb8-cf02845f95c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:29.524833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-aa1acf14-6eb3-43ab-8bb8-cf02845f95c2 None None] MFA Rules not processed for user `455836cdf93740ecb13f33cb6cacc53b`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:29.526052 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-aa1acf14-6eb3-43ab-8bb8-cf02845f95c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:29.595846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1308/2619] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:29.608257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054e39e-743e-4a09-be38-64ef3369aad0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:29.610244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-4054e39e-743e-4a09-be38-64ef3369aad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:29.670610 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054e39e-743e-4a09-be38-64ef3369aad0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:29.674254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88ec35-cf54-42d6-a9e0-0a3a4cb5be52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:29.674902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88ec35-cf54-42d6-a9e0-0a3a4cb5be52 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:29.675190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88ec35-cf54-42d6-a9e0-0a3a4cb5be52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:29.675469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88ec35-cf54-42d6-a9e0-0a3a4cb5be52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:29.738559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88ec35-cf54-42d6-a9e0-0a3a4cb5be52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:29.740018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88ec35-cf54-42d6-a9e0-0a3a4cb5be52 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:29.749051 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1312/2620] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:30.949509 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1acf14-6eb3-43ab-8bb8-cf02845f95c2 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:30.950973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-aa1acf14-6eb3-43ab-8bb8-cf02845f95c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:31.014911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1acf14-6eb3-43ab-8bb8-cf02845f95c2 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:31.019288 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:31.020182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:31.020458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:31.020731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] PATH_INFO: `/v3/users/b7ed9ddfdd6842619789e5adcaa1d6ea` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:31.022019 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7ed9ddfdd6842619789e5adcaa1d6ea)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:31.024520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:31.045918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:31.200148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7ed9ddfdd6842619789e5adcaa1d6ea'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:31.207063 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7ed9ddfdd6842619789e5adcaa1d6ea'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:31.207764 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7ed9ddfdd6842619789e5adcaa1d6ea'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:31.220114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7ed9ddfdd6842619789e5adcaa1d6ea'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:31.220904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7ed9ddfdd6842619789e5adcaa1d6ea'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:31.266386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1309/2621] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:30 2022] DELETE /identity/v3/users/b7ed9ddfdd6842619789e5adcaa1d6ea => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:31.270818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01be329-3e6c-4624-9e31-a7fcdce6010a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:31.271336 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01be329-3e6c-4624-9e31-a7fcdce6010a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:31.271434 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01be329-3e6c-4624-9e31-a7fcdce6010a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:31.290688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-a01be329-3e6c-4624-9e31-a7fcdce6010a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:31.292121 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a01be329-3e6c-4624-9e31-a7fcdce6010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:31.331456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:31.349830 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1313/2622] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:31.360237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:31.361651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:31.418774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec54ab-b11d-417f-9fbf-84e8756b795e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:31.422395 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ef418-2616-48f6-a73a-7ac4a51dc541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:31.423032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef418-2616-48f6-a73a-7ac4a51dc541 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:31.423315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef418-2616-48f6-a73a-7ac4a51dc541 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:31.423590 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef418-2616-48f6-a73a-7ac4a51dc541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:31.433829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ef418-2616-48f6-a73a-7ac4a51dc541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:31.436253 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ef418-2616-48f6-a73a-7ac4a51dc541 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:31.447176 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1310/2623] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:31.889763 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01be329-3e6c-4624-9e31-a7fcdce6010a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:31.891808 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a01be329-3e6c-4624-9e31-a7fcdce6010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:31.907849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01be329-3e6c-4624-9e31-a7fcdce6010a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:31.914401 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:31.915375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:31.915687 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:31.915954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] PATH_INFO: `/v3/projects/66ac7c8ec8bc4299a262f166e9481602` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:31.917266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66ac7c8ec8bc4299a262f166e9481602)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:31.923472 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:32.015395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66ac7c8ec8bc4299a262f166e9481602', 'request_id': 'req-ad174a25-cc83-46e3-ad7b-538c1a68a081'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:32.020996 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66ac7c8ec8bc4299a262f166e9481602', 'request_id': 'req-ad174a25-cc83-46e3-ad7b-538c1a68a081'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:32.028267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66ac7c8ec8bc4299a262f166e9481602', 'request_id': 'req-ad174a25-cc83-46e3-ad7b-538c1a68a081'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:32.098300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] The token cache is being invalidate because project 66ac7c8ec8bc4299a262f166e9481602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:32.102339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:32.102339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:32.102339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1314/2624] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:31 2022] DELETE /identity/v3/projects/66ac7c8ec8bc4299a262f166e9481602 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:32.105297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ef418-2616-48f6-a73a-7ac4a51dc541 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:32.115486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ef418-2616-48f6-a73a-7ac4a51dc541 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:32.172540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:32.173559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:32.173830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:32.174098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] PATH_INFO: `/v3/users/70fd765d1f1e477481cc89c1b2039005` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:32.175290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] RBAC: Authorizing `identity:delete_user(user_id=70fd765d1f1e477481cc89c1b2039005)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:32.177227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:32.341868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:32.344094 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70fd765d1f1e477481cc89c1b2039005'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:32.351650 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70fd765d1f1e477481cc89c1b2039005'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:32.352368 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70fd765d1f1e477481cc89c1b2039005'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:32.365096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70fd765d1f1e477481cc89c1b2039005'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:32.365895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70fd765d1f1e477481cc89c1b2039005'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:32.455198 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1311/2625] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:32 2022] DELETE /identity/v3/users/70fd765d1f1e477481cc89c1b2039005 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:32.842568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:32.843972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:32.856940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad174a25-cc83-46e3-ad7b-538c1a68a081 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:32.886224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:32.889107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:32.922497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:32.923811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:32.924113 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:32.924394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] PATH_INFO: `/v3/users/90cae4f03afd46e4b4bfcb93aec4cd1c` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:32.925655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90cae4f03afd46e4b4bfcb93aec4cd1c)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:32.927721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:32.967061 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd90d6-7af6-4779-9e90-4a06d676dbac admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:32.971743 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:32.972605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:32.972950 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:32.973212 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] PATH_INFO: `/v3/projects/eda08082be72494eb3d90ee8e69388d5` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:32.974442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda08082be72494eb3d90ee8e69388d5)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:32.976423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:33.096158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda08082be72494eb3d90ee8e69388d5', 'request_id': 'req-6151765c-69cc-4597-8c94-62596bf40359'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:33.096573 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90cae4f03afd46e4b4bfcb93aec4cd1c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:33.102822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda08082be72494eb3d90ee8e69388d5', 'request_id': 'req-6151765c-69cc-4597-8c94-62596bf40359'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:33.104200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90cae4f03afd46e4b4bfcb93aec4cd1c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:33.105033 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90cae4f03afd46e4b4bfcb93aec4cd1c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:33.105633 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90cae4f03afd46e4b4bfcb93aec4cd1c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:33.116669 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:33.119449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-50947c72-8d47-48f3-a111-2ed958cc7559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90cae4f03afd46e4b4bfcb93aec4cd1c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:33.195807 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda08082be72494eb3d90ee8e69388d5', 'request_id': 'req-6151765c-69cc-4597-8c94-62596bf40359'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:33.207089 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] The token cache is being invalidate because project eda08082be72494eb3d90ee8e69388d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:33.207377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:33.208026 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-6151765c-69cc-4597-8c94-62596bf40359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:33.209634 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1312/2626] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:32 2022] DELETE /identity/v3/projects/eda08082be72494eb3d90ee8e69388d5 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:33.214170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73adaaa-ac2d-44ac-bdec-4bb7dc96e238 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.214458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73adaaa-ac2d-44ac-bdec-4bb7dc96e238 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.214815 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73adaaa-ac2d-44ac-bdec-4bb7dc96e238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.234024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c73adaaa-ac2d-44ac-bdec-4bb7dc96e238 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:33.235559 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c73adaaa-ac2d-44ac-bdec-4bb7dc96e238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:33.279563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:33.283337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1315/2627] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:32 2022] DELETE /identity/v3/users/90cae4f03afd46e4b4bfcb93aec4cd1c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:33.288307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c26cc-3e42-4ad0-9835-2d7e1c425a98 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.288656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c26cc-3e42-4ad0-9835-2d7e1c425a98 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.289195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c26cc-3e42-4ad0-9835-2d7e1c425a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.295405 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1313/2628] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:33.304935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455881-71b8-4eb1-ba6b-7d83747a1ce3 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.305222 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455881-71b8-4eb1-ba6b-7d83747a1ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.305493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c455881-71b8-4eb1-ba6b-7d83747a1ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.307940 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-482c26cc-3e42-4ad0-9835-2d7e1c425a98 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:33.309196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-482c26cc-3e42-4ad0-9835-2d7e1c425a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:33.322722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-7c455881-71b8-4eb1-ba6b-7d83747a1ce3 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:33.323881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7c455881-71b8-4eb1-ba6b-7d83747a1ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:33.373373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1316/2629] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:33.384792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c26cc-3e42-4ad0-9835-2d7e1c425a98 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:33.386756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-482c26cc-3e42-4ad0-9835-2d7e1c425a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:33.396149 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1314/2630] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:33.400746 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c455881-71b8-4eb1-ba6b-7d83747a1ce3 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:33.409297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c455881-71b8-4eb1-ba6b-7d83747a1ce3 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:33.414553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1edb43-7493-4555-be95-3d3dfe4a3440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:33.415193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1edb43-7493-4555-be95-3d3dfe4a3440 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.415476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1edb43-7493-4555-be95-3d3dfe4a3440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.415749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1edb43-7493-4555-be95-3d3dfe4a3440 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.416467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1edb43-7493-4555-be95-3d3dfe4a3440 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:33.416789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1edb43-7493-4555-be95-3d3dfe4a3440 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:33.418396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1edb43-7493-4555-be95-3d3dfe4a3440 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:33.434235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1315/2631] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:16:33.435494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1edb43-7493-4555-be95-3d3dfe4a3440 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:33.445591 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1edb43-7493-4555-be95-3d3dfe4a3440 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:33.450237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4514cad-1a9c-4ea1-a770-40eccd681b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:33.450873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4514cad-1a9c-4ea1-a770-40eccd681b3e admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.451266 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4514cad-1a9c-4ea1-a770-40eccd681b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.451427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4514cad-1a9c-4ea1-a770-40eccd681b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.452223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4514cad-1a9c-4ea1-a770-40eccd681b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:33.454180 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4514cad-1a9c-4ea1-a770-40eccd681b3e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:33.456584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c26cc-3e42-4ad0-9835-2d7e1c425a98 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:33.460694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2b33e-ae94-4e4c-ac85-25ef95d080ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:33.461344 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2b33e-ae94-4e4c-ac85-25ef95d080ed service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.461632 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2b33e-ae94-4e4c-ac85-25ef95d080ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.461904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2b33e-ae94-4e4c-ac85-25ef95d080ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.471238 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2b33e-ae94-4e4c-ac85-25ef95d080ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:33.472658 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2b33e-ae94-4e4c-ac85-25ef95d080ed service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:33.482282 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1317/2632] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:33.530039 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1316/2633] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:33 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:33.531337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2b33e-ae94-4e4c-ac85-25ef95d080ed service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:33.542259 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2b33e-ae94-4e4c-ac85-25ef95d080ed service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:33.603035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9704e4e-d104-468c-a858-abb7efc02dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:33.603761 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9704e4e-d104-468c-a858-abb7efc02dad admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.604045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9704e4e-d104-468c-a858-abb7efc02dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.604318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9704e4e-d104-468c-a858-abb7efc02dad admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.605086 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9704e4e-d104-468c-a858-abb7efc02dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:33.606992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9704e4e-d104-468c-a858-abb7efc02dad admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:33.701924 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1318/2634] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:33 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:33.706452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4514cad-1a9c-4ea1-a770-40eccd681b3e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:33.717874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4514cad-1a9c-4ea1-a770-40eccd681b3e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:33.722588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ca53f-74e4-4038-8c1b-562f3dbb828e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:33.723270 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ca53f-74e4-4038-8c1b-562f3dbb828e admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.723553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ca53f-74e4-4038-8c1b-562f3dbb828e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.723881 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ca53f-74e4-4038-8c1b-562f3dbb828e admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.724859 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ca53f-74e4-4038-8c1b-562f3dbb828e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:33.727118 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ca53f-74e4-4038-8c1b-562f3dbb828e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:33.741493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1317/2635] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:33.742988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9704e4e-d104-468c-a858-abb7efc02dad admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:33.753489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9704e4e-d104-468c-a858-abb7efc02dad admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:33.758975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10146802-f375-40de-a2fe-ef94fbd15b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:33.759729 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10146802-f375-40de-a2fe-ef94fbd15b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.760961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10146802-f375-40de-a2fe-ef94fbd15b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.760961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10146802-f375-40de-a2fe-ef94fbd15b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.761182 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10146802-f375-40de-a2fe-ef94fbd15b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:33.762666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10146802-f375-40de-a2fe-ef94fbd15b64 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:33.777004 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1319/2636] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:33.778574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ca53f-74e4-4038-8c1b-562f3dbb828e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:33.789786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ca53f-74e4-4038-8c1b-562f3dbb828e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:33.794500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f29150-5ede-42e8-ac93-16fa907a6d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:33.796605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f29150-5ede-42e8-ac93-16fa907a6d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.796605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f29150-5ede-42e8-ac93-16fa907a6d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.796605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f29150-5ede-42e8-ac93-16fa907a6d9c admin admin] PATH_INFO: `/v3/projects/bb81a3c50c98428388f1f0c31b6913de/users/03d5ddd5439f4fbda74c185e9aec16d2/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.810953 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f29150-5ede-42e8-ac93-16fa907a6d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb81a3c50c98428388f1f0c31b6913de, user_id=03d5ddd5439f4fbda74c185e9aec16d2, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:33.813642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f29150-5ede-42e8-ac93-16fa907a6d9c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:33.878713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1318/2637] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:33 2022] PUT /identity/v3/projects/bb81a3c50c98428388f1f0c31b6913de/users/03d5ddd5439f4fbda74c185e9aec16d2/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:33.889419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4bf75-e97c-471d-9ef0-ba65922f5c1b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:33.889914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4bf75-e97c-471d-9ef0-ba65922f5c1b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:33.889914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4bf75-e97c-471d-9ef0-ba65922f5c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:33.908571 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:33.922789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-59c4bf75-e97c-471d-9ef0-ba65922f5c1b None None] MFA Rules not processed for user `03d5ddd5439f4fbda74c185e9aec16d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:33.924007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-59c4bf75-e97c-471d-9ef0-ba65922f5c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:33.980802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f29150-5ede-42e8-ac93-16fa907a6d9c admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:33.982707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-24f29150-5ede-42e8-ac93-16fa907a6d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:33.989618 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1320/2638] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:33.997087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4bf75-e97c-471d-9ef0-ba65922f5c1b None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:34.062046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4bf75-e97c-471d-9ef0-ba65922f5c1b None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:34.066648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616f26f-e43c-444b-9c6d-27e6f6233547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:34.067382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f26f-e43c-444b-9c6d-27e6f6233547 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:34.067613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f26f-e43c-444b-9c6d-27e6f6233547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:34.067897 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f26f-e43c-444b-9c6d-27e6f6233547 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:34.068588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616f26f-e43c-444b-9c6d-27e6f6233547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:34.069855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f29150-5ede-42e8-ac93-16fa907a6d9c admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:34.070595 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616f26f-e43c-444b-9c6d-27e6f6233547 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:34.074449 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:34.075295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:34.075574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:34.075849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] PATH_INFO: `/v3/projects/1126dcf7b2a6440bba4c059edd4320ed` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:34.077096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1126dcf7b2a6440bba4c059edd4320ed)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:34.078977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:34.106191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:34.169887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1321/2639] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:33 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:34.173903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616f26f-e43c-444b-9c6d-27e6f6233547 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:34.209940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1126dcf7b2a6440bba4c059edd4320ed', 'request_id': 'req-b9d37e28-c2f3-4580-8590-854d44d98ccf'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:34.216056 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1126dcf7b2a6440bba4c059edd4320ed', 'request_id': 'req-b9d37e28-c2f3-4580-8590-854d44d98ccf'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:34.236118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616f26f-e43c-444b-9c6d-27e6f6233547 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:34.264276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:34.292631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ea76e-7efa-4340-b57c-bbdd6aa67adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:34.293341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ea76e-7efa-4340-b57c-bbdd6aa67adf admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:34.293650 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ea76e-7efa-4340-b57c-bbdd6aa67adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:34.293997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ea76e-7efa-4340-b57c-bbdd6aa67adf admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:34.294717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ea76e-7efa-4340-b57c-bbdd6aa67adf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:34.295317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1126dcf7b2a6440bba4c059edd4320ed', 'request_id': 'req-b9d37e28-c2f3-4580-8590-854d44d98ccf'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:34.296642 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ea76e-7efa-4340-b57c-bbdd6aa67adf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:34.307215 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] The token cache is being invalidate because project 1126dcf7b2a6440bba4c059edd4320ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:34.307714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:34.308456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:34.310170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1319/2640] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:33 2022] DELETE /identity/v3/projects/1126dcf7b2a6440bba4c059edd4320ed => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:34.315512 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:34.317108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:34.330260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d37e28-c2f3-4580-8590-854d44d98ccf admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:34.334850 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:34.335700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:34.335987 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:34.336261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] PATH_INFO: `/v3/users/455836cdf93740ecb13f33cb6cacc53b` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:34.337466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=455836cdf93740ecb13f33cb6cacc53b)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:34.339363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:34.380866 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1322/2641] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:34 2022] POST /identity/v3/users => generated 298 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:34.385032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ea76e-7efa-4340-b57c-bbdd6aa67adf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:34.387337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b04ea76e-7efa-4340-b57c-bbdd6aa67adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:34.398891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ea76e-7efa-4340-b57c-bbdd6aa67adf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:34.403914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616d2ba-39d3-4c64-a612-d5cb67f04d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:34.404902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616d2ba-39d3-4c64-a612-d5cb67f04d6e admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:34.405214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616d2ba-39d3-4c64-a612-d5cb67f04d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:34.405475 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616d2ba-39d3-4c64-a612-d5cb67f04d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:34.406203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616d2ba-39d3-4c64-a612-d5cb67f04d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:34.407809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616d2ba-39d3-4c64-a612-d5cb67f04d6e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:34.417939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1323/2642] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:34.421816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616d2ba-39d3-4c64-a612-d5cb67f04d6e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:34.431698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616d2ba-39d3-4c64-a612-d5cb67f04d6e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:34.436356 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe4da2-1157-44c9-9d86-10663774ea95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:34.437222 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe4da2-1157-44c9-9d86-10663774ea95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:34.437509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe4da2-1157-44c9-9d86-10663774ea95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:34.437789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe4da2-1157-44c9-9d86-10663774ea95 admin admin] PATH_INFO: `/v3/projects/4cd34cfc4b384ad998888a2d1ac4555d/users/7ff3bdbdb23148b5a896417689a3b6ca/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:34.454050 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe4da2-1157-44c9-9d86-10663774ea95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd34cfc4b384ad998888a2d1ac4555d, user_id=7ff3bdbdb23148b5a896417689a3b6ca, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:34.456734 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe4da2-1157-44c9-9d86-10663774ea95 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:34.496863 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455836cdf93740ecb13f33cb6cacc53b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:34.503404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455836cdf93740ecb13f33cb6cacc53b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:34.504084 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455836cdf93740ecb13f33cb6cacc53b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:34.512746 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1324/2643] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:34 2022] PUT /identity/v3/projects/4cd34cfc4b384ad998888a2d1ac4555d/users/7ff3bdbdb23148b5a896417689a3b6ca/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:34.516634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe4da2-1157-44c9-9d86-10663774ea95 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:34.516916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455836cdf93740ecb13f33cb6cacc53b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:34.517616 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455836cdf93740ecb13f33cb6cacc53b'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:34.518453 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e7fe4da2-1157-44c9-9d86-10663774ea95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:34.582668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe4da2-1157-44c9-9d86-10663774ea95 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:34.586235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1320/2644] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:34 2022] DELETE /identity/v3/users/455836cdf93740ecb13f33cb6cacc53b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:34.592301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1880d82-5dad-41e8-aa6d-6e0093a81bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:34.592301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1880d82-5dad-41e8-aa6d-6e0093a81bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:34.592301 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1880d82-5dad-41e8-aa6d-6e0093a81bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:34.592636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1880d82-5dad-41e8-aa6d-6e0093a81bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:34.593365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1880d82-5dad-41e8-aa6d-6e0093a81bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:34.595020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1880d82-5dad-41e8-aa6d-6e0093a81bc1 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:34.608864 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1325/2645] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:34.610375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:34.681172 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af145fce-e9dd-45ef-a3a5-8b9b2020aa25 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:34.687402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b00f0-7da3-4439-b30a-f7ef0db568e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:34.690214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b00f0-7da3-4439-b30a-f7ef0db568e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:34.690214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b00f0-7da3-4439-b30a-f7ef0db568e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:34.690214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b00f0-7da3-4439-b30a-f7ef0db568e6 admin admin] PATH_INFO: `/v3/projects/4cd34cfc4b384ad998888a2d1ac4555d/users/7ff3bdbdb23148b5a896417689a3b6ca/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:34.692046 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b00f0-7da3-4439-b30a-f7ef0db568e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd34cfc4b384ad998888a2d1ac4555d, user_id=7ff3bdbdb23148b5a896417689a3b6ca, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:34.695249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b00f0-7da3-4439-b30a-f7ef0db568e6 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:34.706108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1321/2646] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:34 2022] PUT /identity/v3/projects/4cd34cfc4b384ad998888a2d1ac4555d/users/7ff3bdbdb23148b5a896417689a3b6ca/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:34.717196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850face-573f-4863-a203-61fb14f44dfa None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:34.717490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850face-573f-4863-a203-61fb14f44dfa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:34.717774 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850face-573f-4863-a203-61fb14f44dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:34.745684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e850face-573f-4863-a203-61fb14f44dfa None None] MFA Rules not processed for user `7ff3bdbdb23148b5a896417689a3b6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:34.746944 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e850face-573f-4863-a203-61fb14f44dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:34.823570 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1326/2647] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:34.836286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b00f0-7da3-4439-b30a-f7ef0db568e6 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:34.838072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d96b00f0-7da3-4439-b30a-f7ef0db568e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:34.869545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:34.912206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b00f0-7da3-4439-b30a-f7ef0db568e6 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:34.916401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8d645-587f-4c5b-9e57-a84ce7fa0b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:34.917102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d645-587f-4c5b-9e57-a84ce7fa0b1c service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:34.917383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d645-587f-4c5b-9e57-a84ce7fa0b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:34.918117 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8d645-587f-4c5b-9e57-a84ce7fa0b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:34.926943 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8d645-587f-4c5b-9e57-a84ce7fa0b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:34.928666 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8d645-587f-4c5b-9e57-a84ce7fa0b1c service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:34.939780 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1322/2648] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:35.066751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850face-573f-4863-a203-61fb14f44dfa None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:35.068186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e850face-573f-4863-a203-61fb14f44dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:35.080970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:35.103561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8d645-587f-4c5b-9e57-a84ce7fa0b1c service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:35.113138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8d645-587f-4c5b-9e57-a84ce7fa0b1c service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:35.116807 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d762c-b441-447e-a0a9-6d036341cae0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:35.117443 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d762c-b441-447e-a0a9-6d036341cae0 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.117725 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d762c-b441-447e-a0a9-6d036341cae0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.118002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d762c-b441-447e-a0a9-6d036341cae0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.119789 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-db7d762c-b441-447e-a0a9-6d036341cae0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:35.146195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850face-573f-4863-a203-61fb14f44dfa None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:35.152026 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:35.153178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.153538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.153955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] PATH_INFO: `/v3/projects/dec9bc42b292433db35c5eb04e7ee7f8` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.155247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dec9bc42b292433db35c5eb04e7ee7f8)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:35.157530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:35.186010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d762c-b441-447e-a0a9-6d036341cae0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:35.187519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d762c-b441-447e-a0a9-6d036341cae0 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:35.197458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1323/2649] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:35.278482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dec9bc42b292433db35c5eb04e7ee7f8', 'request_id': 'req-c9bdfdb8-0afc-4c82-9213-9149371e3b03'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:35.284021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dec9bc42b292433db35c5eb04e7ee7f8', 'request_id': 'req-c9bdfdb8-0afc-4c82-9213-9149371e3b03'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:35.290359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dec9bc42b292433db35c5eb04e7ee7f8', 'request_id': 'req-c9bdfdb8-0afc-4c82-9213-9149371e3b03'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:35.392890 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] The token cache is being invalidate because project dec9bc42b292433db35c5eb04e7ee7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:35.393304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:35.394003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-c9bdfdb8-0afc-4c82-9213-9149371e3b03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:35.395582 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1327/2650] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:35 2022] DELETE /identity/v3/projects/dec9bc42b292433db35c5eb04e7ee7f8 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:35.400660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d06b1-7aa7-402a-93e8-1d9d83b87ca5 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.401161 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d06b1-7aa7-402a-93e8-1d9d83b87ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.401315 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d06b1-7aa7-402a-93e8-1d9d83b87ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.421911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c95d06b1-7aa7-402a-93e8-1d9d83b87ca5 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:35.423112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c95d06b1-7aa7-402a-93e8-1d9d83b87ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:35.483597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1324/2651] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:35.492420 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349be3b-4b4e-447d-bca1-eabce995ba7e None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.492697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349be3b-4b4e-447d-bca1-eabce995ba7e None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.493032 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349be3b-4b4e-447d-bca1-eabce995ba7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.512242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5349be3b-4b4e-447d-bca1-eabce995ba7e None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:35.513499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5349be3b-4b4e-447d-bca1-eabce995ba7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:35.521821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1328/2652] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:35.526793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d06b1-7aa7-402a-93e8-1d9d83b87ca5 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:35.536551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d06b1-7aa7-402a-93e8-1d9d83b87ca5 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:35.541273 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc382ca-ebd1-400c-aebb-48183a1051e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:35.541922 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc382ca-ebd1-400c-aebb-48183a1051e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.542205 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc382ca-ebd1-400c-aebb-48183a1051e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.542511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc382ca-ebd1-400c-aebb-48183a1051e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.543269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc382ca-ebd1-400c-aebb-48183a1051e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:35.543574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc382ca-ebd1-400c-aebb-48183a1051e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:35.545197 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc382ca-ebd1-400c-aebb-48183a1051e4 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:35.557750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1325/2653] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:16:35.562506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349be3b-4b4e-447d-bca1-eabce995ba7e None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:35.572482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349be3b-4b4e-447d-bca1-eabce995ba7e None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:35.577181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99a24-3564-4bc6-a426-eaf6027d7628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:35.577826 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99a24-3564-4bc6-a426-eaf6027d7628 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.578107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99a24-3564-4bc6-a426-eaf6027d7628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.578381 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb99a24-3564-4bc6-a426-eaf6027d7628 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.579062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99a24-3564-4bc6-a426-eaf6027d7628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:35.580977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb99a24-3564-4bc6-a426-eaf6027d7628 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:35.661648 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc382ca-ebd1-400c-aebb-48183a1051e4 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:35.663366 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-3fc382ca-ebd1-400c-aebb-48183a1051e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:35.676202 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc382ca-ebd1-400c-aebb-48183a1051e4 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:35.681968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:35.682848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.683135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.683409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] PATH_INFO: `/v3/users/03d5ddd5439f4fbda74c185e9aec16d2` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.684580 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d5ddd5439f4fbda74c185e9aec16d2)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:35.686256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1329/2654] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:35 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:35.689833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:35.690358 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99a24-3564-4bc6-a426-eaf6027d7628 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:35.752457 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb99a24-3564-4bc6-a426-eaf6027d7628 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:35.757991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d49aa-3aad-4e57-aace-0493ac5b7eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:35.765075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d49aa-3aad-4e57-aace-0493ac5b7eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.765075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d49aa-3aad-4e57-aace-0493ac5b7eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.765075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d49aa-3aad-4e57-aace-0493ac5b7eaf admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.765075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d49aa-3aad-4e57-aace-0493ac5b7eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:35.765075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d49aa-3aad-4e57-aace-0493ac5b7eaf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:35.866686 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:35.873158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1330/2655] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:35 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:35.877308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d49aa-3aad-4e57-aace-0493ac5b7eaf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:35.886878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:35.887299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d49aa-3aad-4e57-aace-0493ac5b7eaf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:35.891751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d5ddd5439f4fbda74c185e9aec16d2'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:35.891913 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e56d66-5a6a-4340-ba04-02058090155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:35.892447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e56d66-5a6a-4340-ba04-02058090155f admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.892731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e56d66-5a6a-4340-ba04-02058090155f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.893038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e56d66-5a6a-4340-ba04-02058090155f admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.893901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e56d66-5a6a-4340-ba04-02058090155f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:35.895498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e56d66-5a6a-4340-ba04-02058090155f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:35.899114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d5ddd5439f4fbda74c185e9aec16d2'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:35.899804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d5ddd5439f4fbda74c185e9aec16d2'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:35.910210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1331/2656] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:35.911070 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e56d66-5a6a-4340-ba04-02058090155f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:35.913597 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-04e56d66-5a6a-4340-ba04-02058090155f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:35.913860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d5ddd5439f4fbda74c185e9aec16d2'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:35.914569 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0e9452b9-bd08-467e-863d-7391b73c86a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d5ddd5439f4fbda74c185e9aec16d2'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:35.927845 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e56d66-5a6a-4340-ba04-02058090155f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:35.932903 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3ab41-fbff-425d-b832-2f507fca9296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:35.933601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ab41-fbff-425d-b832-2f507fca9296 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.933889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ab41-fbff-425d-b832-2f507fca9296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.934169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ab41-fbff-425d-b832-2f507fca9296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.935023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3ab41-fbff-425d-b832-2f507fca9296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:35.936700 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3ab41-fbff-425d-b832-2f507fca9296 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:35.947476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1332/2657] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:35.951786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3ab41-fbff-425d-b832-2f507fca9296 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:35.962700 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3ab41-fbff-425d-b832-2f507fca9296 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:35.967464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc8700-5bde-4670-9409-c37177158333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:35.968353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc8700-5bde-4670-9409-c37177158333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:35.968639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc8700-5bde-4670-9409-c37177158333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:35.972982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc8700-5bde-4670-9409-c37177158333 admin admin] PATH_INFO: `/v3/projects/efa21ec6538f47748099e63ffd137c00/users/ee1e8d63161246e29df65865dc0314cd/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:35.983635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc8700-5bde-4670-9409-c37177158333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efa21ec6538f47748099e63ffd137c00, user_id=ee1e8d63161246e29df65865dc0314cd, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:35.986612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc8700-5bde-4670-9409-c37177158333 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:35.998193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1326/2658] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:35 2022] DELETE /identity/v3/users/03d5ddd5439f4fbda74c185e9aec16d2 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:36.002740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f34499-3a6e-41b8-bbaa-647eea53a3c8 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.003225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f34499-3a6e-41b8-bbaa-647eea53a3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.003700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f34499-3a6e-41b8-bbaa-647eea53a3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.023068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-f0f34499-3a6e-41b8-bbaa-647eea53a3c8 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:36.024358 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f0f34499-3a6e-41b8-bbaa-647eea53a3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:36.061318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1333/2659] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:35 2022] PUT /identity/v3/projects/efa21ec6538f47748099e63ffd137c00/users/ee1e8d63161246e29df65865dc0314cd/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:36.092341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1327/2660] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:36.103292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc8700-5bde-4670-9409-c37177158333 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:36.105346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-eadc8700-5bde-4670-9409-c37177158333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:36.130844 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98548c9-4694-474a-9186-16229f70b92d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.131147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98548c9-4694-474a-9186-16229f70b92d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.131948 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98548c9-4694-474a-9186-16229f70b92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.159572 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-a98548c9-4694-474a-9186-16229f70b92d None None] MFA Rules not processed for user `ee1e8d63161246e29df65865dc0314cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:36.160818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a98548c9-4694-474a-9186-16229f70b92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:36.167598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc8700-5bde-4670-9409-c37177158333 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:36.171192 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd078b25-4217-4b27-a092-cbd7a0a00c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:36.171831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd078b25-4217-4b27-a092-cbd7a0a00c38 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.172111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd078b25-4217-4b27-a092-cbd7a0a00c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.172385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd078b25-4217-4b27-a092-cbd7a0a00c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.224317 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1328/2661] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:36.229447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98548c9-4694-474a-9186-16229f70b92d None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:36.233691 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd078b25-4217-4b27-a092-cbd7a0a00c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:36.235385 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd078b25-4217-4b27-a092-cbd7a0a00c38 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:36.239808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98548c9-4694-474a-9186-16229f70b92d None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:36.245297 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e58c3-48b3-470d-864a-ad22a79b5964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:36.245992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1334/2662] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:36.246140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e58c3-48b3-470d-864a-ad22a79b5964 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.246367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e58c3-48b3-470d-864a-ad22a79b5964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.246647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e58c3-48b3-470d-864a-ad22a79b5964 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.247340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e58c3-48b3-470d-864a-ad22a79b5964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:36.249287 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e58c3-48b3-470d-864a-ad22a79b5964 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:36.354674 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1329/2663] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:36 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:36.358619 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd078b25-4217-4b27-a092-cbd7a0a00c38 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:36.368816 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd078b25-4217-4b27-a092-cbd7a0a00c38 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:36.425186 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12054b-c796-4b0f-a8f0-8863f9140a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:36.425853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12054b-c796-4b0f-a8f0-8863f9140a4f admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.426140 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12054b-c796-4b0f-a8f0-8863f9140a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.426417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12054b-c796-4b0f-a8f0-8863f9140a4f admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.427110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12054b-c796-4b0f-a8f0-8863f9140a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:36.429579 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12054b-c796-4b0f-a8f0-8863f9140a4f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:36.437138 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:36.535440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1335/2664] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:36 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:36.539814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e58c3-48b3-470d-864a-ad22a79b5964 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:36.550455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e58c3-48b3-470d-864a-ad22a79b5964 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:36.555908 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b57fbc-028d-4bba-ba30-d47675dfc185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:36.556799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b57fbc-028d-4bba-ba30-d47675dfc185 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.557103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b57fbc-028d-4bba-ba30-d47675dfc185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.557390 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b57fbc-028d-4bba-ba30-d47675dfc185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.558122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b57fbc-028d-4bba-ba30-d47675dfc185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:36.559759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b57fbc-028d-4bba-ba30-d47675dfc185 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:36.571072 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1330/2665] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:36.575109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12054b-c796-4b0f-a8f0-8863f9140a4f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:36.585318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12054b-c796-4b0f-a8f0-8863f9140a4f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:36.590174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ed847-d610-415e-b3c3-227dae9a69bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:36.590835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ed847-d610-415e-b3c3-227dae9a69bf admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.591122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ed847-d610-415e-b3c3-227dae9a69bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.591400 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ed847-d610-415e-b3c3-227dae9a69bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.592189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ed847-d610-415e-b3c3-227dae9a69bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:36.593841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ed847-d610-415e-b3c3-227dae9a69bf admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:36.606256 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1336/2666] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:36.610898 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b57fbc-028d-4bba-ba30-d47675dfc185 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:36.622654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b57fbc-028d-4bba-ba30-d47675dfc185 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:36.628237 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfaa0e-9f12-4b17-be37-0eaeaecd7c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:36.629345 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfaa0e-9f12-4b17-be37-0eaeaecd7c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.629677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfaa0e-9f12-4b17-be37-0eaeaecd7c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.629952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfaa0e-9f12-4b17-be37-0eaeaecd7c2e admin admin] PATH_INFO: `/v3/projects/e734ca888e564138a899d646b25520ec/users/938bb55f32ac45eb8256bf9eb94d9465/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.642979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:36.646038 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfaa0e-9f12-4b17-be37-0eaeaecd7c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e734ca888e564138a899d646b25520ec, user_id=938bb55f32ac45eb8256bf9eb94d9465, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:36.648661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfaa0e-9f12-4b17-be37-0eaeaecd7c2e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:36.689391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ed847-d610-415e-b3c3-227dae9a69bf admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:36.692072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-751ed847-d610-415e-b3c3-227dae9a69bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:36.706752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ed847-d610-415e-b3c3-227dae9a69bf admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:36.711696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:36.712655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.714249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.714249 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] PATH_INFO: `/v3/projects/bb81a3c50c98428388f1f0c31b6913de` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.714648 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb81a3c50c98428388f1f0c31b6913de)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:36.716568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:36.732206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1331/2667] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:36 2022] PUT /identity/v3/projects/e734ca888e564138a899d646b25520ec/users/938bb55f32ac45eb8256bf9eb94d9465/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:36.743471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a559a0-9e80-44ca-a30f-46a6b6ae4843 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.743970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a559a0-9e80-44ca-a30f-46a6b6ae4843 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.744399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a559a0-9e80-44ca-a30f-46a6b6ae4843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.773108 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-82a559a0-9e80-44ca-a30f-46a6b6ae4843 None None] MFA Rules not processed for user `938bb55f32ac45eb8256bf9eb94d9465`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:36.774472 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-82a559a0-9e80-44ca-a30f-46a6b6ae4843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:36.827430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb81a3c50c98428388f1f0c31b6913de', 'request_id': 'req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:36.833447 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb81a3c50c98428388f1f0c31b6913de', 'request_id': 'req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:36.840608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1332/2668] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:36.841044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb81a3c50c98428388f1f0c31b6913de', 'request_id': 'req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:36.845122 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a559a0-9e80-44ca-a30f-46a6b6ae4843 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:36.908406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a559a0-9e80-44ca-a30f-46a6b6ae4843 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:36.923671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] The token cache is being invalidate because project bb81a3c50c98428388f1f0c31b6913de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:36.924055 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:36.924795 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:36.929979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1337/2669] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:36 2022] DELETE /identity/v3/projects/bb81a3c50c98428388f1f0c31b6913de => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:36.931023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:36.932375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:36.978294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c073c6-b2ac-4ae1-9762-c474ee2205b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:36.979836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c073c6-b2ac-4ae1-9762-c474ee2205b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:36.980128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c073c6-b2ac-4ae1-9762-c474ee2205b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:36.980406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c073c6-b2ac-4ae1-9762-c474ee2205b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:36.981150 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c073c6-b2ac-4ae1-9762-c474ee2205b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:36.983104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c073c6-b2ac-4ae1-9762-c474ee2205b3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:37.003392 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b594a7-a3c7-4ad4-8c8f-7fbb3b07f350 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:37.008639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:37.009542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:37.009831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:37.010112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] PATH_INFO: `/v3/users/7ff3bdbdb23148b5a896417689a3b6ca` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:37.011314 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff3bdbdb23148b5a896417689a3b6ca)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:37.013275 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:37.077078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1333/2670] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:36 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:37.078032 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c073c6-b2ac-4ae1-9762-c474ee2205b3 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:37.079644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d3c073c6-b2ac-4ae1-9762-c474ee2205b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:37.093933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:37.155738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c073c6-b2ac-4ae1-9762-c474ee2205b3 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:37.161191 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77229077-7f0d-47ce-9d46-5b41809f6126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:37.161978 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77229077-7f0d-47ce-9d46-5b41809f6126 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:37.162337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77229077-7f0d-47ce-9d46-5b41809f6126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:37.163260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77229077-7f0d-47ce-9d46-5b41809f6126 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:37.167092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77229077-7f0d-47ce-9d46-5b41809f6126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:37.167092 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77229077-7f0d-47ce-9d46-5b41809f6126 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:37.213410 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff3bdbdb23148b5a896417689a3b6ca'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:37.224007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff3bdbdb23148b5a896417689a3b6ca'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:37.226108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff3bdbdb23148b5a896417689a3b6ca'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:37.227021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff3bdbdb23148b5a896417689a3b6ca'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:37.237538 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=3,keystone:SELECT=82,keystone:DELETE=15 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:37.243894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff3bdbdb23148b5a896417689a3b6ca'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:37.281839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1334/2671] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:37 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:37.282901 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77229077-7f0d-47ce-9d46-5b41809f6126 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:37.284606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-77229077-7f0d-47ce-9d46-5b41809f6126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:37.298311 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77229077-7f0d-47ce-9d46-5b41809f6126 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:37.305087 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e37f8-0857-4d9b-a4d6-18a98dc2dc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:37.305798 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e37f8-0857-4d9b-a4d6-18a98dc2dc0a admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:37.328013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e37f8-0857-4d9b-a4d6-18a98dc2dc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:37.328013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e37f8-0857-4d9b-a4d6-18a98dc2dc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:37.328013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e37f8-0857-4d9b-a4d6-18a98dc2dc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:37.328013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e37f8-0857-4d9b-a4d6-18a98dc2dc0a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:37.328013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1335/2672] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:37.330912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e37f8-0857-4d9b-a4d6-18a98dc2dc0a admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:37.341169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e37f8-0857-4d9b-a4d6-18a98dc2dc0a admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:37.345973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09786a19-8787-4351-a56f-5769e4e82854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:37.346842 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09786a19-8787-4351-a56f-5769e4e82854 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:37.347124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09786a19-8787-4351-a56f-5769e4e82854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:37.347396 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09786a19-8787-4351-a56f-5769e4e82854 admin admin] PATH_INFO: `/v3/projects/6380b63f3ea2441ea24cf9c5b56a0c4a/users/95617d91837b461ea42ae55a022fb670/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:37.363088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09786a19-8787-4351-a56f-5769e4e82854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6380b63f3ea2441ea24cf9c5b56a0c4a, user_id=95617d91837b461ea42ae55a022fb670, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:37.366833 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09786a19-8787-4351-a56f-5769e4e82854 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:37.374927 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1338/2673] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:36 2022] DELETE /identity/v3/users/7ff3bdbdb23148b5a896417689a3b6ca => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:37.439146 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1336/2674] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:37 2022] PUT /identity/v3/projects/6380b63f3ea2441ea24cf9c5b56a0c4a/users/95617d91837b461ea42ae55a022fb670/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:37.443375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:37.506203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e346b-ff0f-4f77-ad9b-257c019f266f admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:37.560745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fc972-5b55-4fec-996a-7a4553a46d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:37.561515 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc972-5b55-4fec-996a-7a4553a46d5b admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:37.561861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc972-5b55-4fec-996a-7a4553a46d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:37.562203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc972-5b55-4fec-996a-7a4553a46d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:37.563035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fc972-5b55-4fec-996a-7a4553a46d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:37.564735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fc972-5b55-4fec-996a-7a4553a46d5b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:37.576490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1339/2675] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:37.580751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09786a19-8787-4351-a56f-5769e4e82854 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:37.592207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09786a19-8787-4351-a56f-5769e4e82854 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:37.597036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0cce6-b911-4ec1-acef-3eeb0f6c2987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:37.597931 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cce6-b911-4ec1-acef-3eeb0f6c2987 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:37.598236 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cce6-b911-4ec1-acef-3eeb0f6c2987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:37.598511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cce6-b911-4ec1-acef-3eeb0f6c2987 admin admin] PATH_INFO: `/v3/projects/6380b63f3ea2441ea24cf9c5b56a0c4a/users/95617d91837b461ea42ae55a022fb670/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:37.601213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0cce6-b911-4ec1-acef-3eeb0f6c2987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6380b63f3ea2441ea24cf9c5b56a0c4a, user_id=95617d91837b461ea42ae55a022fb670, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:37.603815 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0cce6-b911-4ec1-acef-3eeb0f6c2987 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:37.615240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1337/2676] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:37 2022] PUT /identity/v3/projects/6380b63f3ea2441ea24cf9c5b56a0c4a/users/95617d91837b461ea42ae55a022fb670/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:37.627373 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eea0ce-cac7-4fd1-b968-46b3050b4e23 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:37.627666 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eea0ce-cac7-4fd1-b968-46b3050b4e23 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:37.627942 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eea0ce-cac7-4fd1-b968-46b3050b4e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:37.654841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-67eea0ce-cac7-4fd1-b968-46b3050b4e23 None None] MFA Rules not processed for user `95617d91837b461ea42ae55a022fb670`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:37.656066 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-67eea0ce-cac7-4fd1-b968-46b3050b4e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:37.725449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1340/2677] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:37.737466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0cce6-b911-4ec1-acef-3eeb0f6c2987 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:37.739147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-95b0cce6-b911-4ec1-acef-3eeb0f6c2987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:37.765378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eea0ce-cac7-4fd1-b968-46b3050b4e23 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:37.766856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-67eea0ce-cac7-4fd1-b968-46b3050b4e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:37.808119 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0cce6-b911-4ec1-acef-3eeb0f6c2987 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:37.811884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac615e34-e131-4b9c-8e86-7d2895e732bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:37.812526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac615e34-e131-4b9c-8e86-7d2895e732bd service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:37.812837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac615e34-e131-4b9c-8e86-7d2895e732bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:37.813117 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac615e34-e131-4b9c-8e86-7d2895e732bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:37.815173 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ac615e34-e131-4b9c-8e86-7d2895e732bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:37.833731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eea0ce-cac7-4fd1-b968-46b3050b4e23 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:37.839741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:37.840601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:37.841395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:37.841678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] PATH_INFO: `/v3/projects/4cd34cfc4b384ad998888a2d1ac4555d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:37.843394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd34cfc4b384ad998888a2d1ac4555d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:37.845787 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:37.848417 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:37.875768 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac615e34-e131-4b9c-8e86-7d2895e732bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:37.877238 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac615e34-e131-4b9c-8e86-7d2895e732bd service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:37.886342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1338/2678] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:37.954489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd34cfc4b384ad998888a2d1ac4555d', 'request_id': 'req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:37.960566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd34cfc4b384ad998888a2d1ac4555d', 'request_id': 'req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:37.966991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd34cfc4b384ad998888a2d1ac4555d', 'request_id': 'req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:38.060183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] The token cache is being invalidate because project 4cd34cfc4b384ad998888a2d1ac4555d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:38.060488 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:38.061212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b2f01ac7-d598-4dfc-bc3f-e7c8a155bee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:38.062878 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1341/2679] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:37 2022] DELETE /identity/v3/projects/4cd34cfc4b384ad998888a2d1ac4555d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:38.067681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fcc2c-9521-46ea-affd-0fc7eb088ec9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:38.067970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fcc2c-9521-46ea-affd-0fc7eb088ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:38.068246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fcc2c-9521-46ea-affd-0fc7eb088ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:38.088291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b23fcc2c-9521-46ea-affd-0fc7eb088ec9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:38.089573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b23fcc2c-9521-46ea-affd-0fc7eb088ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:38.094677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:38.150535 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1339/2680] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:38.213721 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968a5da-e6b6-4212-816f-afbdb3550c95 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:38.214296 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968a5da-e6b6-4212-816f-afbdb3550c95 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:38.214296 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968a5da-e6b6-4212-816f-afbdb3550c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:38.232699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3968a5da-e6b6-4212-816f-afbdb3550c95 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:38.233929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3968a5da-e6b6-4212-816f-afbdb3550c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:38.242760 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1342/2681] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:38.247651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fcc2c-9521-46ea-affd-0fc7eb088ec9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:38.257422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fcc2c-9521-46ea-affd-0fc7eb088ec9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:38.261921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb5265-ca6e-4339-bd7b-32af35e5d59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:38.262564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5265-ca6e-4339-bd7b-32af35e5d59c admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:38.262853 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5265-ca6e-4339-bd7b-32af35e5d59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:38.263129 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5265-ca6e-4339-bd7b-32af35e5d59c admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:38.263858 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb5265-ca6e-4339-bd7b-32af35e5d59c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:38.264163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb5265-ca6e-4339-bd7b-32af35e5d59c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:38.265797 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb5265-ca6e-4339-bd7b-32af35e5d59c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:38.278781 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1340/2682] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:16:38.283379 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968a5da-e6b6-4212-816f-afbdb3550c95 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:38.293419 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968a5da-e6b6-4212-816f-afbdb3550c95 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:38.298242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50401826-0dfa-49cf-a703-3d0b4d5015fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:38.299022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50401826-0dfa-49cf-a703-3d0b4d5015fa admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:38.299517 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50401826-0dfa-49cf-a703-3d0b4d5015fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:38.299880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50401826-0dfa-49cf-a703-3d0b4d5015fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:38.300718 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50401826-0dfa-49cf-a703-3d0b4d5015fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:38.302847 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50401826-0dfa-49cf-a703-3d0b4d5015fa admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:38.383576 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1343/2683] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:38 2022] POST /identity/v3/projects => generated 359 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:38.387890 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb5265-ca6e-4339-bd7b-32af35e5d59c admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:38.398241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb5265-ca6e-4339-bd7b-32af35e5d59c admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:38.457736 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaad613-54ec-4d4f-a040-63f672a4c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:38.458404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaad613-54ec-4d4f-a040-63f672a4c473 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:38.458689 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaad613-54ec-4d4f-a040-63f672a4c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:38.458967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaad613-54ec-4d4f-a040-63f672a4c473 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:38.459692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaad613-54ec-4d4f-a040-63f672a4c473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:38.461638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaad613-54ec-4d4f-a040-63f672a4c473 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:38.563673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1341/2684] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:38 2022] POST /identity/v3/users => generated 280 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:38.571591 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50401826-0dfa-49cf-a703-3d0b4d5015fa admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:38.581953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50401826-0dfa-49cf-a703-3d0b4d5015fa admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:38.587190 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be654f-9540-4927-88bf-907b6dc99646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:38.587849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be654f-9540-4927-88bf-907b6dc99646 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:38.588144 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be654f-9540-4927-88bf-907b6dc99646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:38.588437 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be654f-9540-4927-88bf-907b6dc99646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:38.589198 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be654f-9540-4927-88bf-907b6dc99646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:38.590833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be654f-9540-4927-88bf-907b6dc99646 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:38.602257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1344/2685] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:38.606023 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaad613-54ec-4d4f-a040-63f672a4c473 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:38.617071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaad613-54ec-4d4f-a040-63f672a4c473 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:38.632275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537b467-f874-4a9d-851a-874d3af6dd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:38.632275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537b467-f874-4a9d-851a-874d3af6dd35 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:38.632275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537b467-f874-4a9d-851a-874d3af6dd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:38.632275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537b467-f874-4a9d-851a-874d3af6dd35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:38.632861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537b467-f874-4a9d-851a-874d3af6dd35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:38.632861 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537b467-f874-4a9d-851a-874d3af6dd35 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:38.641664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1342/2686] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:38.646181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be654f-9540-4927-88bf-907b6dc99646 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:38.658688 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be654f-9540-4927-88bf-907b6dc99646 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:38.665316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74290bb7-3564-4aa5-a713-88ab65daac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:38.666231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74290bb7-3564-4aa5-a713-88ab65daac24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:38.666662 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74290bb7-3564-4aa5-a713-88ab65daac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:38.666780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74290bb7-3564-4aa5-a713-88ab65daac24 admin admin] PATH_INFO: `/v3/projects/7fc7a150755d43bfbcf56732eddb2739/users/b2ca0825e63a46d290ac45a54161cf4c/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:38.683877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74290bb7-3564-4aa5-a713-88ab65daac24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc7a150755d43bfbcf56732eddb2739, user_id=b2ca0825e63a46d290ac45a54161cf4c, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:38.686628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74290bb7-3564-4aa5-a713-88ab65daac24 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:38.758813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1345/2687] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:38 2022] PUT /identity/v3/projects/7fc7a150755d43bfbcf56732eddb2739/users/b2ca0825e63a46d290ac45a54161cf4c/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:38.769575 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b04b95-ffbc-4d52-87fe-2e947d6a6ba7 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:38.769962 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b04b95-ffbc-4d52-87fe-2e947d6a6ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:38.770247 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b04b95-ffbc-4d52-87fe-2e947d6a6ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:38.797663 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-a1b04b95-ffbc-4d52-87fe-2e947d6a6ba7 None None] MFA Rules not processed for user `b2ca0825e63a46d290ac45a54161cf4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:38.798876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a1b04b95-ffbc-4d52-87fe-2e947d6a6ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:38.862716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1343/2688] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:38 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:38.894167 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74290bb7-3564-4aa5-a713-88ab65daac24 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:38.895877 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-74290bb7-3564-4aa5-a713-88ab65daac24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:38.946199 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:38.963213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74290bb7-3564-4aa5-a713-88ab65daac24 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:38.967054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc1117-0294-46d2-84a0-1088242ac551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:38.967756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc1117-0294-46d2-84a0-1088242ac551 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:38.967980 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc1117-0294-46d2-84a0-1088242ac551 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:38.968257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc1117-0294-46d2-84a0-1088242ac551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:38.977667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc1117-0294-46d2-84a0-1088242ac551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:38.979202 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc1117-0294-46d2-84a0-1088242ac551 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:38.993074 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1346/2689] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:38 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:39.009154 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b04b95-ffbc-4d52-87fe-2e947d6a6ba7 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:39.024452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:39.082980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b04b95-ffbc-4d52-87fe-2e947d6a6ba7 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:39.088113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:39.089083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:39.089296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:39.089561 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] PATH_INFO: `/v3/users/b2ca0825e63a46d290ac45a54161cf4c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:39.090823 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ca0825e63a46d290ac45a54161cf4c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:39.092739 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:39.347555 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ca0825e63a46d290ac45a54161cf4c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:39.355171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ca0825e63a46d290ac45a54161cf4c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:39.355871 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ca0825e63a46d290ac45a54161cf4c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:39.368102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ca0825e63a46d290ac45a54161cf4c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:39.368750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ca0825e63a46d290ac45a54161cf4c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:39.447777 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1344/2690] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:39 2022] DELETE /identity/v3/users/b2ca0825e63a46d290ac45a54161cf4c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:39.452206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35709daf-e5f4-431e-ade4-64b5bbb0cb17 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:39.453068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35709daf-e5f4-431e-ade4-64b5bbb0cb17 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:39.453355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35709daf-e5f4-431e-ade4-64b5bbb0cb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:39.471899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-35709daf-e5f4-431e-ade4-64b5bbb0cb17 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:39.473216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-35709daf-e5f4-431e-ade4-64b5bbb0cb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:39.533861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1347/2691] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:39.544148 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:39.545601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:39.613562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e26e47-53d0-4163-ba86-2f851e3376c4 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:39.618375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761454-34fb-4d8c-82df-44d94472dcd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:39.619035 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761454-34fb-4d8c-82df-44d94472dcd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:39.619404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761454-34fb-4d8c-82df-44d94472dcd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:39.619727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761454-34fb-4d8c-82df-44d94472dcd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:39.638475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a761454-34fb-4d8c-82df-44d94472dcd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:39.639914 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a761454-34fb-4d8c-82df-44d94472dcd8 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:39.649869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1345/2692] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:39.870073 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35709daf-e5f4-431e-ade4-64b5bbb0cb17 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:39.871520 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-35709daf-e5f4-431e-ade4-64b5bbb0cb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:39.931097 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35709daf-e5f4-431e-ade4-64b5bbb0cb17 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:39.934720 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43af2a-58cb-464c-8066-3a7e438f14b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:39.935368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43af2a-58cb-464c-8066-3a7e438f14b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:39.935653 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43af2a-58cb-464c-8066-3a7e438f14b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:39.935932 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43af2a-58cb-464c-8066-3a7e438f14b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:39.937751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1e43af2a-58cb-464c-8066-3a7e438f14b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:39.998434 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43af2a-58cb-464c-8066-3a7e438f14b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:39.999807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43af2a-58cb-464c-8066-3a7e438f14b7 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:40.009359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1348/2693] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:40.062382 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761454-34fb-4d8c-82df-44d94472dcd8 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:40.063799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-6a761454-34fb-4d8c-82df-44d94472dcd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:40.078584 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761454-34fb-4d8c-82df-44d94472dcd8 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:40.083419 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:40.084438 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:40.084827 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:40.085235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] PATH_INFO: `/v3/projects/7fc7a150755d43bfbcf56732eddb2739` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:40.086507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc7a150755d43bfbcf56732eddb2739)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:40.089024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:40.115030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:40.210582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc7a150755d43bfbcf56732eddb2739', 'request_id': 'req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:40.216794 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc7a150755d43bfbcf56732eddb2739', 'request_id': 'req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:40.300422 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc7a150755d43bfbcf56732eddb2739', 'request_id': 'req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:40.314370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] The token cache is being invalidate because project 7fc7a150755d43bfbcf56732eddb2739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:40.314755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:40.315430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:40.325517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1346/2694] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:40 2022] DELETE /identity/v3/projects/7fc7a150755d43bfbcf56732eddb2739 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:41.233091 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43af2a-58cb-464c-8066-3a7e438f14b7 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:41.234568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1e43af2a-58cb-464c-8066-3a7e438f14b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:41.251119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43af2a-58cb-464c-8066-3a7e438f14b7 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:41.285545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:41.302875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfb18b-95be-4358-83f2-27f6ef3c188f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:41.303637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfb18b-95be-4358-83f2-27f6ef3c188f service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:41.303997 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfb18b-95be-4358-83f2-27f6ef3c188f service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:41.304347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfb18b-95be-4358-83f2-27f6ef3c188f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:41.306391 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b8dfb18b-95be-4358-83f2-27f6ef3c188f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:41.376138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:41.377675 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:41.386551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfb18b-95be-4358-83f2-27f6ef3c188f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:41.386551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfb18b-95be-4358-83f2-27f6ef3c188f service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:41.396346 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1349/2695] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:41.438517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c99f83-2f32-4a7a-a40d-a53737fe1e6c admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:41.442228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f0e42-f107-4c57-af94-87f7cd26a9f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:41.442940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f0e42-f107-4c57-af94-87f7cd26a9f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:41.449485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f0e42-f107-4c57-af94-87f7cd26a9f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:41.449485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f0e42-f107-4c57-af94-87f7cd26a9f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:41.449485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-992f0e42-f107-4c57-af94-87f7cd26a9f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:41.515009 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f0e42-f107-4c57-af94-87f7cd26a9f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:41.516458 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f0e42-f107-4c57-af94-87f7cd26a9f7 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:41.526239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1347/2696] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:46.125386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfb18b-95be-4358-83f2-27f6ef3c188f service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:46.126931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b8dfb18b-95be-4358-83f2-27f6ef3c188f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:46.138245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfb18b-95be-4358-83f2-27f6ef3c188f service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:46.141786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95d0ce-96e5-452a-8328-ad7f89ffcfa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:46.142416 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95d0ce-96e5-452a-8328-ad7f89ffcfa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:46.142695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95d0ce-96e5-452a-8328-ad7f89ffcfa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:46.142967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95d0ce-96e5-452a-8328-ad7f89ffcfa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:46.144670 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0f95d0ce-96e5-452a-8328-ad7f89ffcfa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:46.206894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95d0ce-96e5-452a-8328-ad7f89ffcfa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:46.208322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95d0ce-96e5-452a-8328-ad7f89ffcfa9 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:46.218290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1350/2697] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:46.286767 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f0e42-f107-4c57-af94-87f7cd26a9f7 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:46.288808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-992f0e42-f107-4c57-af94-87f7cd26a9f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:46.301813 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f0e42-f107-4c57-af94-87f7cd26a9f7 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:46.306333 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:46.307194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:46.307479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:46.307755 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] PATH_INFO: `/v3/users/ee1e8d63161246e29df65865dc0314cd` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:46.308969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee1e8d63161246e29df65865dc0314cd)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:46.310878 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:46.472476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee1e8d63161246e29df65865dc0314cd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:46.478672 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:46.480951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee1e8d63161246e29df65865dc0314cd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:46.481721 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee1e8d63161246e29df65865dc0314cd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:46.495268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee1e8d63161246e29df65865dc0314cd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:46.495925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee1e8d63161246e29df65865dc0314cd'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:46.580915 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1348/2698] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:46 2022] DELETE /identity/v3/users/ee1e8d63161246e29df65865dc0314cd => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:46.585792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c50d6-c2f4-40c4-9580-4168bae4e3fa None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:46.586089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c50d6-c2f4-40c4-9580-4168bae4e3fa None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:46.588409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c50d6-c2f4-40c4-9580-4168bae4e3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:46.605308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-1c7c50d6-c2f4-40c4-9580-4168bae4e3fa None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:46.606592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1c7c50d6-c2f4-40c4-9580-4168bae4e3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:46.668420 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1351/2699] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:46.678996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:46.680468 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:46.739312 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdad17-ba0d-4aa0-a8de-7881f71f35d3 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:46.743722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad8c34-b2b6-4712-ad5e-3a449210ac4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:46.744372 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad8c34-b2b6-4712-ad5e-3a449210ac4d service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:46.744658 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad8c34-b2b6-4712-ad5e-3a449210ac4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:46.744966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad8c34-b2b6-4712-ad5e-3a449210ac4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:46.754313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad8c34-b2b6-4712-ad5e-3a449210ac4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:46.755802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad8c34-b2b6-4712-ad5e-3a449210ac4d service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:46.765954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1349/2700] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:47.198856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c50d6-c2f4-40c4-9580-4168bae4e3fa None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:47.200841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1c7c50d6-c2f4-40c4-9580-4168bae4e3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:47.216174 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c50d6-c2f4-40c4-9580-4168bae4e3fa None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:47.221219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:47.222046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:47.222324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:47.222738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] PATH_INFO: `/v3/projects/efa21ec6538f47748099e63ffd137c00` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:47.223979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efa21ec6538f47748099e63ffd137c00)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:47.226817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:47.247724 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:47.360635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efa21ec6538f47748099e63ffd137c00', 'request_id': 'req-78b45303-278c-45d9-a1e2-11efce5204e7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:47.366897 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efa21ec6538f47748099e63ffd137c00', 'request_id': 'req-78b45303-278c-45d9-a1e2-11efce5204e7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:47.373630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efa21ec6538f47748099e63ffd137c00', 'request_id': 'req-78b45303-278c-45d9-a1e2-11efce5204e7'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:47.469733 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] The token cache is being invalidate because project efa21ec6538f47748099e63ffd137c00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:47.470049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:47.470679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:47.472269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1352/2701] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:47 2022] DELETE /identity/v3/projects/efa21ec6538f47748099e63ffd137c00 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:47.476869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad8c34-b2b6-4712-ad5e-3a449210ac4d service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:47.487538 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad8c34-b2b6-4712-ad5e-3a449210ac4d service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:47.544211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:47.545124 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:47.545414 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:47.545699 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] PATH_INFO: `/v3/users/938bb55f32ac45eb8256bf9eb94d9465` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:47.546900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=938bb55f32ac45eb8256bf9eb94d9465)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:47.549885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:47.749910 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '938bb55f32ac45eb8256bf9eb94d9465'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:47.757920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '938bb55f32ac45eb8256bf9eb94d9465'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:47.758735 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '938bb55f32ac45eb8256bf9eb94d9465'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:47.771179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '938bb55f32ac45eb8256bf9eb94d9465'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:47.772381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '938bb55f32ac45eb8256bf9eb94d9465'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:47.775946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:47.849440 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1350/2702] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:47 2022] DELETE /identity/v3/users/938bb55f32ac45eb8256bf9eb94d9465 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:48.500723 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:48.502807 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:48.516639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b45303-278c-45d9-a1e2-11efce5204e7 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:48.597260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:48.598124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:48.598414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:48.598696 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] PATH_INFO: `/v3/projects/e734ca888e564138a899d646b25520ec` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:48.601263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e734ca888e564138a899d646b25520ec)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:48.603355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:48.763559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e734ca888e564138a899d646b25520ec', 'request_id': 'req-f7cac0e5-72e9-4516-8a1c-0e32521a493c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:48.769607 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e734ca888e564138a899d646b25520ec', 'request_id': 'req-f7cac0e5-72e9-4516-8a1c-0e32521a493c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:48.777048 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e734ca888e564138a899d646b25520ec', 'request_id': 'req-f7cac0e5-72e9-4516-8a1c-0e32521a493c'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:48.869624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] The token cache is being invalidate because project e734ca888e564138a899d646b25520ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:48.871023 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:48.871254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:48.873214 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1353/2703] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:48 2022] DELETE /identity/v3/projects/e734ca888e564138a899d646b25520ec => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:48.877610 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:48.879698 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:48.897722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4d46b-a6a2-4fab-88f7-48b2a346b0e8 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:48.960137 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:48.961335 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:48.961722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:48.962098 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] PATH_INFO: `/v3/users/95617d91837b461ea42ae55a022fb670` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:48.963408 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] RBAC: Authorizing `identity:delete_user(user_id=95617d91837b461ea42ae55a022fb670)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:48.965478 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:49.149717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95617d91837b461ea42ae55a022fb670'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:49.158955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95617d91837b461ea42ae55a022fb670'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:49.159685 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95617d91837b461ea42ae55a022fb670'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:49.172447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95617d91837b461ea42ae55a022fb670'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:49.173155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95617d91837b461ea42ae55a022fb670'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:49.266291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1351/2704] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:48 2022] DELETE /identity/v3/users/95617d91837b461ea42ae55a022fb670 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:49.878699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:49.880973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:49.898979 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cac0e5-72e9-4516-8a1c-0e32521a493c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:49.934609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:49.969371 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:49.970341 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:49.970631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:49.970914 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] PATH_INFO: `/v3/projects/6380b63f3ea2441ea24cf9c5b56a0c4a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:49.972206 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6380b63f3ea2441ea24cf9c5b56a0c4a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:49.974263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:50.079622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6380b63f3ea2441ea24cf9c5b56a0c4a', 'request_id': 'req-e9000fb4-5420-4b39-8ca3-dcb2b184780f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.085323 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6380b63f3ea2441ea24cf9c5b56a0c4a', 'request_id': 'req-e9000fb4-5420-4b39-8ca3-dcb2b184780f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.091826 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6380b63f3ea2441ea24cf9c5b56a0c4a', 'request_id': 'req-e9000fb4-5420-4b39-8ca3-dcb2b184780f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.159944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:50.161819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:50.180068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc888475-8293-4b4d-951a-6d73d026152e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:50.186635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] The token cache is being invalidate because project 6380b63f3ea2441ea24cf9c5b56a0c4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:50.186635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.186635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-e9000fb4-5420-4b39-8ca3-dcb2b184780f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.186635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1354/2705] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:49 2022] DELETE /identity/v3/projects/6380b63f3ea2441ea24cf9c5b56a0c4a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:50.190225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72106761-89f1-44d8-826f-9f28217dad96 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.190465 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72106761-89f1-44d8-826f-9f28217dad96 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.190744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72106761-89f1-44d8-826f-9f28217dad96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.215555 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-72106761-89f1-44d8-826f-9f28217dad96 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:50.215920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-72106761-89f1-44d8-826f-9f28217dad96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:50.246873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:50.247769 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.249226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.249226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] PATH_INFO: `/v3/users/d7e2fc1ec676438392d7925b43b806f5` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.249574 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7e2fc1ec676438392d7925b43b806f5)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:50.251520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:50.265619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:50.283856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1355/2706] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:50.292037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76912c9-7036-45dc-96ff-1219b5bf1326 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.292333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76912c9-7036-45dc-96ff-1219b5bf1326 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.292612 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76912c9-7036-45dc-96ff-1219b5bf1326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.311108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-d76912c9-7036-45dc-96ff-1219b5bf1326 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:50.312163 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-d76912c9-7036-45dc-96ff-1219b5bf1326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:50.320007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1356/2707] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:50.323929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76912c9-7036-45dc-96ff-1219b5bf1326 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:50.332553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76912c9-7036-45dc-96ff-1219b5bf1326 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:50.337187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd897b56-02e5-4a10-81d8-371e24df75a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:50.337805 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd897b56-02e5-4a10-81d8-371e24df75a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.344990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd897b56-02e5-4a10-81d8-371e24df75a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.344990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd897b56-02e5-4a10-81d8-371e24df75a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.344990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd897b56-02e5-4a10-81d8-371e24df75a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:50.344990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd897b56-02e5-4a10-81d8-371e24df75a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:50.344990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd897b56-02e5-4a10-81d8-371e24df75a6 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:50.352784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1357/2708] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:16:50.356737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd897b56-02e5-4a10-81d8-371e24df75a6 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:50.366777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd897b56-02e5-4a10-81d8-371e24df75a6 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:50.371261 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412de70-e3e7-43a0-b647-bee45aa5587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:50.371860 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412de70-e3e7-43a0-b647-bee45aa5587d admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.372143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412de70-e3e7-43a0-b647-bee45aa5587d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.372422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412de70-e3e7-43a0-b647-bee45aa5587d admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.373129 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412de70-e3e7-43a0-b647-bee45aa5587d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:50.374934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412de70-e3e7-43a0-b647-bee45aa5587d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:50.417224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7e2fc1ec676438392d7925b43b806f5'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.424132 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7e2fc1ec676438392d7925b43b806f5'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.424869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7e2fc1ec676438392d7925b43b806f5'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.437402 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7e2fc1ec676438392d7925b43b806f5'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.443061 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7e2fc1ec676438392d7925b43b806f5'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.496224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1358/2709] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:50 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:50.500311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412de70-e3e7-43a0-b647-bee45aa5587d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:50.502116 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0412de70-e3e7-43a0-b647-bee45aa5587d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:50.522657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1352/2710] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:16:50 2022] DELETE /identity/v3/users/d7e2fc1ec676438392d7925b43b806f5 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:50.529131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:50.571685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412de70-e3e7-43a0-b647-bee45aa5587d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:50.625844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444dc2f-6d60-40b6-a824-ae814c502386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:50.626757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444dc2f-6d60-40b6-a824-ae814c502386 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.627046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444dc2f-6d60-40b6-a824-ae814c502386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.627388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444dc2f-6d60-40b6-a824-ae814c502386 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.628499 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444dc2f-6d60-40b6-a824-ae814c502386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:50.630695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444dc2f-6d60-40b6-a824-ae814c502386 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:50.724187 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1359/2711] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:50 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:50.728727 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:50.738964 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f9895-c81a-49c5-91bd-1bf3efc57c42 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:50.743456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434583e-c153-43d9-bd25-39336d298627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:50.744224 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434583e-c153-43d9-bd25-39336d298627 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.744512 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434583e-c153-43d9-bd25-39336d298627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.744814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434583e-c153-43d9-bd25-39336d298627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.745537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434583e-c153-43d9-bd25-39336d298627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:50.747147 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434583e-c153-43d9-bd25-39336d298627 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:50.757058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1353/2712] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:50.760910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444dc2f-6d60-40b6-a824-ae814c502386 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:50.771736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444dc2f-6d60-40b6-a824-ae814c502386 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:50.776426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53dfe4e-4d09-45df-b03f-5fce196c2900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:50.777112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53dfe4e-4d09-45df-b03f-5fce196c2900 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.777399 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53dfe4e-4d09-45df-b03f-5fce196c2900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.777677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53dfe4e-4d09-45df-b03f-5fce196c2900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.778521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53dfe4e-4d09-45df-b03f-5fce196c2900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:50.780119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53dfe4e-4d09-45df-b03f-5fce196c2900 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:50.790689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1360/2713] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:50.794712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434583e-c153-43d9-bd25-39336d298627 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:50.804581 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434583e-c153-43d9-bd25-39336d298627 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:50.809188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caff740-3535-4005-bad5-dbee61b098a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:50.810049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caff740-3535-4005-bad5-dbee61b098a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.810330 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caff740-3535-4005-bad5-dbee61b098a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.810604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caff740-3535-4005-bad5-dbee61b098a5 admin admin] PATH_INFO: `/v3/projects/1879d1d27e5443b09c8b339cdbb6b774/users/9b72f7d5ea8541d09f3a5a1fb643bce5/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.821652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53dfe4e-4d09-45df-b03f-5fce196c2900 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:50.823929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b53dfe4e-4d09-45df-b03f-5fce196c2900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:50.827670 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caff740-3535-4005-bad5-dbee61b098a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1879d1d27e5443b09c8b339cdbb6b774, user_id=9b72f7d5ea8541d09f3a5a1fb643bce5, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:50.830985 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caff740-3535-4005-bad5-dbee61b098a5 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:50.839871 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53dfe4e-4d09-45df-b03f-5fce196c2900 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:50.844694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:50.845562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.845848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.846124 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] PATH_INFO: `/v3/projects/3ec37a457530439b8728dd2ad2b6b7bd` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.847319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec37a457530439b8728dd2ad2b6b7bd)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:50.850471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:50.934268 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1354/2714] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:50 2022] PUT /identity/v3/projects/1879d1d27e5443b09c8b339cdbb6b774/users/9b72f7d5ea8541d09f3a5a1fb643bce5/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:50.944891 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b419a-0274-48ce-8149-745f137c86f8 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:50.945185 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b419a-0274-48ce-8149-745f137c86f8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:50.945480 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b419a-0274-48ce-8149-745f137c86f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:50.969722 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec37a457530439b8728dd2ad2b6b7bd', 'request_id': 'req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.970790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-d66b419a-0274-48ce-8149-745f137c86f8 None None] MFA Rules not processed for user `9b72f7d5ea8541d09f3a5a1fb643bce5`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:50.971996 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d66b419a-0274-48ce-8149-745f137c86f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:50.976009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec37a457530439b8728dd2ad2b6b7bd', 'request_id': 'req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:50.982792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec37a457530439b8728dd2ad2b6b7bd', 'request_id': 'req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:51.036637 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1355/2715] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:51.041563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b419a-0274-48ce-8149-745f137c86f8 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.054638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] The token cache is being invalidate because project 3ec37a457530439b8728dd2ad2b6b7bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:16:51.054945 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:51.056046 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-523d0884-f7c2-4e74-9067-2b1d9c52e8a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:16:51.057880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1361/2716] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:16:50 2022] DELETE /identity/v3/projects/3ec37a457530439b8728dd2ad2b6b7bd => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:51.067482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079893be-5195-44e7-8a2d-c64fe7541e76 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.067844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079893be-5195-44e7-8a2d-c64fe7541e76 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.068211 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079893be-5195-44e7-8a2d-c64fe7541e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.086411 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-079893be-5195-44e7-8a2d-c64fe7541e76 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:51.087553 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-079893be-5195-44e7-8a2d-c64fe7541e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:51.089179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:51.104357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b419a-0274-48ce-8149-745f137c86f8 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.108974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07140e-8e05-46dd-acc6-83a515a5b757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.109630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07140e-8e05-46dd-acc6-83a515a5b757 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.110190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07140e-8e05-46dd-acc6-83a515a5b757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.110462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07140e-8e05-46dd-acc6-83a515a5b757 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.111181 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07140e-8e05-46dd-acc6-83a515a5b757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.113111 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07140e-8e05-46dd-acc6-83a515a5b757 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.114717 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:51.148427 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1362/2717] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:51.156058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf9061-acb7-46fd-afbd-25b098079879 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.156347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf9061-acb7-46fd-afbd-25b098079879 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.156626 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf9061-acb7-46fd-afbd-25b098079879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.174435 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3bbf9061-acb7-46fd-afbd-25b098079879 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:51.175820 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3bbf9061-acb7-46fd-afbd-25b098079879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:51.184150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1363/2718] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:51.188059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf9061-acb7-46fd-afbd-25b098079879 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.193530 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1356/2719] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:51 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:51.196925 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf9061-acb7-46fd-afbd-25b098079879 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.197074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07140e-8e05-46dd-acc6-83a515a5b757 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.199276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-5b07140e-8e05-46dd-acc6-83a515a5b757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:51.254387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304998c5-7020-4433-8752-9623d45c9ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.255135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304998c5-7020-4433-8752-9623d45c9ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.255438 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304998c5-7020-4433-8752-9623d45c9ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.256327 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304998c5-7020-4433-8752-9623d45c9ba0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.257112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304998c5-7020-4433-8752-9623d45c9ba0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:16:51.257422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304998c5-7020-4433-8752-9623d45c9ba0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.259057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304998c5-7020-4433-8752-9623d45c9ba0 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.261388 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07140e-8e05-46dd-acc6-83a515a5b757 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.265993 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9c078-cc1a-4cf7-8204-7690ed02b69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.266651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9c078-cc1a-4cf7-8204-7690ed02b69c admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.266934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9c078-cc1a-4cf7-8204-7690ed02b69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.267208 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9c078-cc1a-4cf7-8204-7690ed02b69c admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.267890 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9c078-cc1a-4cf7-8204-7690ed02b69c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.269975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9c078-cc1a-4cf7-8204-7690ed02b69c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.275137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1364/2720] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:16:51.277812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304998c5-7020-4433-8752-9623d45c9ba0 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.287980 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304998c5-7020-4433-8752-9623d45c9ba0 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.292835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464ecb7-52ed-4628-9737-0059789bb239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.293497 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464ecb7-52ed-4628-9737-0059789bb239 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.293784 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464ecb7-52ed-4628-9737-0059789bb239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.294063 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464ecb7-52ed-4628-9737-0059789bb239 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.294754 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464ecb7-52ed-4628-9737-0059789bb239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.296757 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464ecb7-52ed-4628-9737-0059789bb239 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.374790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1357/2721] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:51 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:51.378514 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9c078-cc1a-4cf7-8204-7690ed02b69c admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.388562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9c078-cc1a-4cf7-8204-7690ed02b69c admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.393107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c599e-c855-4cde-9b38-1c4fd04d4310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.393732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c599e-c855-4cde-9b38-1c4fd04d4310 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.394014 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c599e-c855-4cde-9b38-1c4fd04d4310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.394290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c599e-c855-4cde-9b38-1c4fd04d4310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.394997 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c599e-c855-4cde-9b38-1c4fd04d4310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.396597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c599e-c855-4cde-9b38-1c4fd04d4310 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.409257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1365/2722] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:51 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:51.409527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1358/2723] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:51.410510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c599e-c855-4cde-9b38-1c4fd04d4310 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.410635 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464ecb7-52ed-4628-9737-0059789bb239 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.420340 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c599e-c855-4cde-9b38-1c4fd04d4310 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.472711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464ecb7-52ed-4628-9737-0059789bb239 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.477425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3da51-9481-42cf-a603-2d92a9453e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.478058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede8ac-13ca-4d6f-9420-e8c36731fc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.478196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3da51-9481-42cf-a603-2d92a9453e94 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.478368 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3da51-9481-42cf-a603-2d92a9453e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.478644 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3da51-9481-42cf-a603-2d92a9453e94 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.478753 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede8ac-13ca-4d6f-9420-e8c36731fc26 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.478989 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede8ac-13ca-4d6f-9420-e8c36731fc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.479265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede8ac-13ca-4d6f-9420-e8c36731fc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.479369 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3da51-9481-42cf-a603-2d92a9453e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.479992 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede8ac-13ca-4d6f-9420-e8c36731fc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.481295 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3da51-9481-42cf-a603-2d92a9453e94 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.481622 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede8ac-13ca-4d6f-9420-e8c36731fc26 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.493471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1359/2724] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:51.497847 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede8ac-13ca-4d6f-9420-e8c36731fc26 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.507519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede8ac-13ca-4d6f-9420-e8c36731fc26 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.512102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d9a8b-83e4-4849-a16e-1d69e7802053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.512968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d9a8b-83e4-4849-a16e-1d69e7802053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.513249 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d9a8b-83e4-4849-a16e-1d69e7802053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.513524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d9a8b-83e4-4849-a16e-1d69e7802053 admin admin] PATH_INFO: `/v3/projects/198fea43540348d2a9c53021e1c767c0/users/61e74fc7e34441c08c63fad6a1dcc58f/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.528380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d9a8b-83e4-4849-a16e-1d69e7802053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=198fea43540348d2a9c53021e1c767c0, user_id=61e74fc7e34441c08c63fad6a1dcc58f, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.531055 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d9a8b-83e4-4849-a16e-1d69e7802053 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.575838 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1366/2725] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:51 2022] POST /identity/v3/users => generated 283 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:16:51.579093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3da51-9481-42cf-a603-2d92a9453e94 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.588986 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3da51-9481-42cf-a603-2d92a9453e94 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.593711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63295025-83a9-4c1e-8132-a32ec70e6a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.594337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63295025-83a9-4c1e-8132-a32ec70e6a02 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.594624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63295025-83a9-4c1e-8132-a32ec70e6a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.594904 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63295025-83a9-4c1e-8132-a32ec70e6a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.595726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63295025-83a9-4c1e-8132-a32ec70e6a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.597615 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63295025-83a9-4c1e-8132-a32ec70e6a02 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.603587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1360/2726] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:51 2022] PUT /identity/v3/projects/198fea43540348d2a9c53021e1c767c0/users/61e74fc7e34441c08c63fad6a1dcc58f/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:51.611324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1367/2727] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:16:51.616860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d9a8b-83e4-4849-a16e-1d69e7802053 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.617305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0e569-f33a-47e4-9b0a-74138283491f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.617601 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0e569-f33a-47e4-9b0a-74138283491f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.617984 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0e569-f33a-47e4-9b0a-74138283491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.642485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:51.644575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-b5c0e569-f33a-47e4-9b0a-74138283491f None None] MFA Rules not processed for user `61e74fc7e34441c08c63fad6a1dcc58f`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:51.645834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b5c0e569-f33a-47e4-9b0a-74138283491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:51.650480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:51.681002 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d9a8b-83e4-4849-a16e-1d69e7802053 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.685757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267641f-e7b9-4d10-87d7-13d20a1607c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.686420 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267641f-e7b9-4d10-87d7-13d20a1607c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.686711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267641f-e7b9-4d10-87d7-13d20a1607c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.687371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267641f-e7b9-4d10-87d7-13d20a1607c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.687876 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267641f-e7b9-4d10-87d7-13d20a1607c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.690141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267641f-e7b9-4d10-87d7-13d20a1607c6 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.701793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1361/2728] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:51.705829 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267641f-e7b9-4d10-87d7-13d20a1607c6 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.715602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1368/2729] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:51.716213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267641f-e7b9-4d10-87d7-13d20a1607c6 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.721652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0e569-f33a-47e4-9b0a-74138283491f None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.721828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34918a0a-27c7-45ee-beff-9cc2d18ba9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.721828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34918a0a-27c7-45ee-beff-9cc2d18ba9a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.722040 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34918a0a-27c7-45ee-beff-9cc2d18ba9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.722216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34918a0a-27c7-45ee-beff-9cc2d18ba9a6 admin admin] PATH_INFO: `/v3/projects/4b5443e46ef94a2e8a065c88e943b027/users/325a3e8931e0495d865802c4700a5ec2/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.726442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0e569-f33a-47e4-9b0a-74138283491f None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.731141 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3a64d-bb1d-43cb-9b0c-f273188822e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.731786 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3a64d-bb1d-43cb-9b0c-f273188822e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.732079 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3a64d-bb1d-43cb-9b0c-f273188822e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.732355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3a64d-bb1d-43cb-9b0c-f273188822e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.733068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3a64d-bb1d-43cb-9b0c-f273188822e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.735234 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3a64d-bb1d-43cb-9b0c-f273188822e8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.738495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34918a0a-27c7-45ee-beff-9cc2d18ba9a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5443e46ef94a2e8a065c88e943b027, user_id=325a3e8931e0495d865802c4700a5ec2, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.741226 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34918a0a-27c7-45ee-beff-9cc2d18ba9a6 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:51.847400 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1362/2730] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:51 2022] PUT /identity/v3/projects/4b5443e46ef94a2e8a065c88e943b027/users/325a3e8931e0495d865802c4700a5ec2/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:51.858204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670cb52-eeee-42ca-ac7a-06d8a0d4eae9 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.858495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670cb52-eeee-42ca-ac7a-06d8a0d4eae9 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.858775 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670cb52-eeee-42ca-ac7a-06d8a0d4eae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.877703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1369/2731] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:16:51 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:51.882870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3a64d-bb1d-43cb-9b0c-f273188822e8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:51.950456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3a64d-bb1d-43cb-9b0c-f273188822e8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:51.955282 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d70b3-31d9-4e98-8b03-0eb0c069906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:51.955982 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d70b3-31d9-4e98-8b03-0eb0c069906e admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:51.956276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d70b3-31d9-4e98-8b03-0eb0c069906e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:51.956557 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d70b3-31d9-4e98-8b03-0eb0c069906e admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:51.957297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d70b3-31d9-4e98-8b03-0eb0c069906e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:51.959357 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d70b3-31d9-4e98-8b03-0eb0c069906e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:52.013287 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1670cb52-eeee-42ca-ac7a-06d8a0d4eae9 None None] MFA Rules not processed for user `325a3e8931e0495d865802c4700a5ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:52.014536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1670cb52-eeee-42ca-ac7a-06d8a0d4eae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:52.043980 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1370/2732] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:16:51 2022] POST /identity/v3/users => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:16:52.047861 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d70b3-31d9-4e98-8b03-0eb0c069906e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:52.058515 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d70b3-31d9-4e98-8b03-0eb0c069906e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:52.063453 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd51992-0d8e-4bc4-bd5d-8a2ec8822cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:52.064111 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd51992-0d8e-4bc4-bd5d-8a2ec8822cad admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:52.064397 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd51992-0d8e-4bc4-bd5d-8a2ec8822cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:52.064680 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd51992-0d8e-4bc4-bd5d-8a2ec8822cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:52.065606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd51992-0d8e-4bc4-bd5d-8a2ec8822cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:52.067276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd51992-0d8e-4bc4-bd5d-8a2ec8822cad admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:52.080299 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1371/2733] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:52.081028 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1363/2734] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:16:52.081584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd51992-0d8e-4bc4-bd5d-8a2ec8822cad admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:52.092286 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd51992-0d8e-4bc4-bd5d-8a2ec8822cad admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:52.092912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670cb52-eeee-42ca-ac7a-06d8a0d4eae9 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:52.094289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1670cb52-eeee-42ca-ac7a-06d8a0d4eae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:52.097277 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba311-3a6d-4e72-b148-310d644956e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:52.098150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba311-3a6d-4e72-b148-310d644956e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:52.098432 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba311-3a6d-4e72-b148-310d644956e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:52.098707 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba311-3a6d-4e72-b148-310d644956e4 admin admin] PATH_INFO: `/v3/projects/e024abc8063e488395a29967e633cd61/users/a35b3865b9744f8baff449df0b6b015f/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:52.114961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ba311-3a6d-4e72-b148-310d644956e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e024abc8063e488395a29967e633cd61, user_id=a35b3865b9744f8baff449df0b6b015f, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:52.117695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ba311-3a6d-4e72-b148-310d644956e4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:52.151916 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670cb52-eeee-42ca-ac7a-06d8a0d4eae9 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:52.155873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1ecc6-0920-4fc6-b242-6912dfece4d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:52.160758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1ecc6-0920-4fc6-b242-6912dfece4d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:52.160758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1ecc6-0920-4fc6-b242-6912dfece4d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:52.160758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1ecc6-0920-4fc6-b242-6912dfece4d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:52.166130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1ecc6-0920-4fc6-b242-6912dfece4d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:52.167583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1ecc6-0920-4fc6-b242-6912dfece4d1 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:52.176275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1364/2735] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:16:52.186409 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1372/2736] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:52 2022] PUT /identity/v3/projects/e024abc8063e488395a29967e633cd61/users/a35b3865b9744f8baff449df0b6b015f/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:16:52.192830 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1ecc6-0920-4fc6-b242-6912dfece4d1 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:52.201074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1ecc6-0920-4fc6-b242-6912dfece4d1 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:52.226585 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:52.260493 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fc2d6-a3d1-4ded-934f-670f4fbe936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:52.260627 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fc2d6-a3d1-4ded-934f-670f4fbe936d admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:52.260866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fc2d6-a3d1-4ded-934f-670f4fbe936d admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:52.261151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fc2d6-a3d1-4ded-934f-670f4fbe936d admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:52.261875 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fc2d6-a3d1-4ded-934f-670f4fbe936d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:52.263460 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fc2d6-a3d1-4ded-934f-670f4fbe936d admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:52.277712 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1365/2737] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:16:52.282061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba311-3a6d-4e72-b148-310d644956e4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:52.292665 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba311-3a6d-4e72-b148-310d644956e4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:52.297355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31f9f2-40c3-4e33-8daf-504bf999d03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:52.298225 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31f9f2-40c3-4e33-8daf-504bf999d03d admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:52.298745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31f9f2-40c3-4e33-8daf-504bf999d03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:52.298745 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31f9f2-40c3-4e33-8daf-504bf999d03d admin admin] PATH_INFO: `/v3/projects/e024abc8063e488395a29967e633cd61/users/a35b3865b9744f8baff449df0b6b015f/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:52.300741 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31f9f2-40c3-4e33-8daf-504bf999d03d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e024abc8063e488395a29967e633cd61, user_id=a35b3865b9744f8baff449df0b6b015f, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:52.304149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31f9f2-40c3-4e33-8daf-504bf999d03d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:52.314778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1373/2738] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:16:52 2022] PUT /identity/v3/projects/e024abc8063e488395a29967e633cd61/users/a35b3865b9744f8baff449df0b6b015f/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:16:52.325500 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeedc06-7ba3-4507-a099-79210c38a939 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:52.325795 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeedc06-7ba3-4507-a099-79210c38a939 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:52.326085 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeedc06-7ba3-4507-a099-79210c38a939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:52.352309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-ffeedc06-7ba3-4507-a099-79210c38a939 None None] MFA Rules not processed for user `a35b3865b9744f8baff449df0b6b015f`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:16:52.353533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ffeedc06-7ba3-4507-a099-79210c38a939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:52.420425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1366/2739] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:16:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:16:52.432242 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31f9f2-40c3-4e33-8daf-504bf999d03d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:52.434021 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1c31f9f2-40c3-4e33-8daf-504bf999d03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:52.462286 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:16:52.492703 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31f9f2-40c3-4e33-8daf-504bf999d03d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:52.496331 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060d96-7392-457f-8540-b892c51b485b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:52.497011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060d96-7392-457f-8540-b892c51b485b service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:52.497298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060d96-7392-457f-8540-b892c51b485b service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:52.497578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060d96-7392-457f-8540-b892c51b485b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:52.499442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0c060d96-7392-457f-8540-b892c51b485b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:16:52.558179 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c060d96-7392-457f-8540-b892c51b485b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:52.559613 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c060d96-7392-457f-8540-b892c51b485b service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:52.568889 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1374/2740] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:54.537304 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeedc06-7ba3-4507-a099-79210c38a939 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:54.546804 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeedc06-7ba3-4507-a099-79210c38a939 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:54.550266 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8688f49-479c-4848-9122-9db433b0baf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:54.550895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8688f49-479c-4848-9122-9db433b0baf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:54.551179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8688f49-479c-4848-9122-9db433b0baf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:54.551457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8688f49-479c-4848-9122-9db433b0baf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:54.608696 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8688f49-479c-4848-9122-9db433b0baf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:54.610429 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8688f49-479c-4848-9122-9db433b0baf4 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:54.628245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1367/2741] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:16:56.130330 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060d96-7392-457f-8540-b892c51b485b service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:16:56.139209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060d96-7392-457f-8540-b892c51b485b service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:16:56.142730 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8987d-7192-49fe-9296-be08f149c54e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:16:56.143360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8987d-7192-49fe-9296-be08f149c54e service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:16:56.143643 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8987d-7192-49fe-9296-be08f149c54e service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:16:56.143915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8987d-7192-49fe-9296-be08f149c54e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:16:56.153197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8987d-7192-49fe-9296-be08f149c54e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:16:56.155750 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8987d-7192-49fe-9296-be08f149c54e service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:16:56.164862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1375/2742] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:16:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:00.641321 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8688f49-479c-4848-9122-9db433b0baf4 service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:00.690123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=100 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:00.704731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8688f49-479c-4848-9122-9db433b0baf4 service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:00.709369 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:00.710241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:00.710526 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:00.710831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] PATH_INFO: `/v3/users/9b72f7d5ea8541d09f3a5a1fb643bce5` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:00.712677 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b72f7d5ea8541d09f3a5a1fb643bce5)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:00.715447 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:00.937416 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b72f7d5ea8541d09f3a5a1fb643bce5'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:00.944720 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b72f7d5ea8541d09f3a5a1fb643bce5'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:00.945428 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b72f7d5ea8541d09f3a5a1fb643bce5'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:00.958120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b72f7d5ea8541d09f3a5a1fb643bce5'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:00.958800 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b72f7d5ea8541d09f3a5a1fb643bce5'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:00.972540 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8987d-7192-49fe-9296-be08f149c54e service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:00.973988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3ef8987d-7192-49fe-9296-be08f149c54e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:01.047205 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8987d-7192-49fe-9296-be08f149c54e service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:01.052054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0637d-0226-4f2e-baf0-e0809d62bef4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:01.052723 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0637d-0226-4f2e-baf0-e0809d62bef4 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:01.053078 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0637d-0226-4f2e-baf0-e0809d62bef4 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:01.053351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0637d-0226-4f2e-baf0-e0809d62bef4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:01.055240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5ed0637d-0226-4f2e-baf0-e0809d62bef4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:01.066702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0637d-0226-4f2e-baf0-e0809d62bef4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:01.068316 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0637d-0226-4f2e-baf0-e0809d62bef4 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:01.078609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1376/2743] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:01.082635 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1368/2744] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:17:00 2022] DELETE /identity/v3/users/9b72f7d5ea8541d09f3a5a1fb643bce5 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:01.087531 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b029e-aa12-44ac-800f-5a2080080455 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:01.087819 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b029e-aa12-44ac-800f-5a2080080455 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:01.093263 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b029e-aa12-44ac-800f-5a2080080455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:01.106382 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-395b029e-aa12-44ac-800f-5a2080080455 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:01.107578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-395b029e-aa12-44ac-800f-5a2080080455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:01.158068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=100 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:01.169771 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1377/2745] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:17:01.179527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:01.180783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:01.243826 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8f3c9-8283-4a1c-96f8-f822e251a24f admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:01.247476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92666647-08e5-4afd-b12f-8b92c2f227ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:01.248114 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92666647-08e5-4afd-b12f-8b92c2f227ec service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:01.248393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92666647-08e5-4afd-b12f-8b92c2f227ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:01.248667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92666647-08e5-4afd-b12f-8b92c2f227ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:01.257942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92666647-08e5-4afd-b12f-8b92c2f227ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:01.259527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92666647-08e5-4afd-b12f-8b92c2f227ec service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:01.269187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1369/2746] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:01.735938 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b029e-aa12-44ac-800f-5a2080080455 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:01.737466 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-395b029e-aa12-44ac-800f-5a2080080455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:01.750636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b029e-aa12-44ac-800f-5a2080080455 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:01.755527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:01.756408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:01.756694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:01.757013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] PATH_INFO: `/v3/projects/1879d1d27e5443b09c8b339cdbb6b774` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:01.758231 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=1879d1d27e5443b09c8b339cdbb6b774)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:01.760172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:01.882065 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1879d1d27e5443b09c8b339cdbb6b774', 'request_id': 'req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:01.888043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1879d1d27e5443b09c8b339cdbb6b774', 'request_id': 'req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:01.894441 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1879d1d27e5443b09c8b339cdbb6b774', 'request_id': 'req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:01.965967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] The token cache is being invalidate because project 1879d1d27e5443b09c8b339cdbb6b774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:17:01.966250 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:01.966854 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:01.968443 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1378/2747] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:17:01 2022] DELETE /identity/v3/projects/1879d1d27e5443b09c8b339cdbb6b774 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:01.972461 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92666647-08e5-4afd-b12f-8b92c2f227ec service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:01.982076 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92666647-08e5-4afd-b12f-8b92c2f227ec service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:02.038136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:02.039612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:02.039896 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:02.040171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] PATH_INFO: `/v3/users/61e74fc7e34441c08c63fad6a1dcc58f` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:02.041377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61e74fc7e34441c08c63fad6a1dcc58f)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:02.043324 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:02.053841 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:02.066746 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:02.068213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:02.081907 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e82ab-c4de-42a1-ac46-87b35c7bf6ec admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:02.087283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:02.091812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:02.091812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:02.091812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] PATH_INFO: `/v3/users/325a3e8931e0495d865802c4700a5ec2` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:02.091812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=325a3e8931e0495d865802c4700a5ec2)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:02.091812 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:02.231895 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61e74fc7e34441c08c63fad6a1dcc58f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.239142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61e74fc7e34441c08c63fad6a1dcc58f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.239785 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61e74fc7e34441c08c63fad6a1dcc58f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.251714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61e74fc7e34441c08c63fad6a1dcc58f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.252336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-ea64f958-f181-4b1f-843a-36bc4885f950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61e74fc7e34441c08c63fad6a1dcc58f'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.324157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325a3e8931e0495d865802c4700a5ec2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.330963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325a3e8931e0495d865802c4700a5ec2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.331656 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325a3e8931e0495d865802c4700a5ec2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.332247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325a3e8931e0495d865802c4700a5ec2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.343982 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1370/2748] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:17:01 2022] DELETE /identity/v3/users/61e74fc7e34441c08c63fad6a1dcc58f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:02.347826 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325a3e8931e0495d865802c4700a5ec2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.416230 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1379/2749] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:17:02 2022] DELETE /identity/v3/users/325a3e8931e0495d865802c4700a5ec2 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:02.418104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9660280-e0c4-42f1-ae20-cd74e1dcf698 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:02.418470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9660280-e0c4-42f1-ae20-cd74e1dcf698 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:02.418822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9660280-e0c4-42f1-ae20-cd74e1dcf698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:02.439604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c9660280-e0c4-42f1-ae20-cd74e1dcf698 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:02.440873 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c9660280-e0c4-42f1-ae20-cd74e1dcf698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:02.499387 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1371/2750] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:02.509010 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:02.510423 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:02.570269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695ae10-b28b-49ea-9e3e-6fbae1010633 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:02.574269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74385e3-14fc-4e30-89b5-7e2bcfc8a44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:02.575060 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74385e3-14fc-4e30-89b5-7e2bcfc8a44e service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:02.575442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74385e3-14fc-4e30-89b5-7e2bcfc8a44e service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:02.575920 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74385e3-14fc-4e30-89b5-7e2bcfc8a44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:02.584887 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:02.589244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74385e3-14fc-4e30-89b5-7e2bcfc8a44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:02.590790 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74385e3-14fc-4e30-89b5-7e2bcfc8a44e service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:02.600500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1380/2751] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:02.821839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9660280-e0c4-42f1-ae20-cd74e1dcf698 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:02.823342 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c9660280-e0c4-42f1-ae20-cd74e1dcf698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:02.835482 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9660280-e0c4-42f1-ae20-cd74e1dcf698 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:02.840438 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:02.841520 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:02.849328 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:02.855171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] PATH_INFO: `/v3/projects/198fea43540348d2a9c53021e1c767c0` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:02.855171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=198fea43540348d2a9c53021e1c767c0)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:02.855171 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:02.981824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '198fea43540348d2a9c53021e1c767c0', 'request_id': 'req-914413f0-c0d3-4124-aff2-ee09b1232a4e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:02.988981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '198fea43540348d2a9c53021e1c767c0', 'request_id': 'req-914413f0-c0d3-4124-aff2-ee09b1232a4e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.048378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '198fea43540348d2a9c53021e1c767c0', 'request_id': 'req-914413f0-c0d3-4124-aff2-ee09b1232a4e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.060194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] The token cache is being invalidate because project 198fea43540348d2a9c53021e1c767c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:17:03.060503 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.061132 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.062722 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1372/2752] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:17:02 2022] DELETE /identity/v3/projects/198fea43540348d2a9c53021e1c767c0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:17:03.066661 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74385e3-14fc-4e30-89b5-7e2bcfc8a44e service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:03.076822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74385e3-14fc-4e30-89b5-7e2bcfc8a44e service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:03.089350 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:03.094518 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:03.107836 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:03.145228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:03.146108 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:03.146394 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:03.146671 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] PATH_INFO: `/v3/users/a35b3865b9744f8baff449df0b6b015f` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:03.147931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a35b3865b9744f8baff449df0b6b015f)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:03.150598 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:03.163294 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914413f0-c0d3-4124-aff2-ee09b1232a4e admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:03.168186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:03.169186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:03.169470 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:03.169743 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] PATH_INFO: `/v3/projects/4b5443e46ef94a2e8a065c88e943b027` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:03.170945 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5443e46ef94a2e8a065c88e943b027)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:03.173374 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:03.319899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5443e46ef94a2e8a065c88e943b027', 'request_id': 'req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.321031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a35b3865b9744f8baff449df0b6b015f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.327216 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5443e46ef94a2e8a065c88e943b027', 'request_id': 'req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.328031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a35b3865b9744f8baff449df0b6b015f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.328719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a35b3865b9744f8baff449df0b6b015f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.329342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a35b3865b9744f8baff449df0b6b015f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.341169 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6d405c25-a7c2-492d-b8a7-ff0db918a773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a35b3865b9744f8baff449df0b6b015f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.385495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5443e46ef94a2e8a065c88e943b027', 'request_id': 'req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.396101 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] The token cache is being invalidate because project 4b5443e46ef94a2e8a065c88e943b027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:17:03.396411 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.397062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-4e54ee66-59d9-4ef1-bb8b-ef6cc3f542a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:03.398599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1373/2753] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:17:03 2022] DELETE /identity/v3/projects/4b5443e46ef94a2e8a065c88e943b027 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:17:03.403244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829845f-2089-4232-963f-9c03f7329bfb None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:03.403534 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829845f-2089-4232-963f-9c03f7329bfb None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:03.403810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829845f-2089-4232-963f-9c03f7329bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:03.420779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-a829845f-2089-4232-963f-9c03f7329bfb None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:03.421969 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a829845f-2089-4232-963f-9c03f7329bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:03.453912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1381/2754] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:17:03 2022] DELETE /identity/v3/users/a35b3865b9744f8baff449df0b6b015f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:03.497647 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1374/2755] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:17:03.499516 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36167f4b-c132-449b-9a40-a9db67793761 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:03.499813 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36167f4b-c132-449b-9a40-a9db67793761 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:03.500095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36167f4b-c132-449b-9a40-a9db67793761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:03.519246 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-36167f4b-c132-449b-9a40-a9db67793761 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:03.520464 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-36167f4b-c132-449b-9a40-a9db67793761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:03.580934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1382/2756] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:03.585223 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829845f-2089-4232-963f-9c03f7329bfb None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:03.595069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829845f-2089-4232-963f-9c03f7329bfb None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:03.599596 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c604d-fbe4-427d-8075-9bb53b1f7782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:03.600265 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c604d-fbe4-427d-8075-9bb53b1f7782 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:03.600554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c604d-fbe4-427d-8075-9bb53b1f7782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:03.600852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c604d-fbe4-427d-8075-9bb53b1f7782 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:03.601582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c604d-fbe4-427d-8075-9bb53b1f7782 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:17:03.601894 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c604d-fbe4-427d-8075-9bb53b1f7782 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:03.603475 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c604d-fbe4-427d-8075-9bb53b1f7782 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:03.615261 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1375/2757] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:17:03.620197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36167f4b-c132-449b-9a40-a9db67793761 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:03.630684 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36167f4b-c132-449b-9a40-a9db67793761 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:03.636037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c95ffb-5f18-45ad-9ac3-5b63694612c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:03.636695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c95ffb-5f18-45ad-9ac3-5b63694612c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:03.637013 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c95ffb-5f18-45ad-9ac3-5b63694612c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:03.637337 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c95ffb-5f18-45ad-9ac3-5b63694612c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:03.637999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c95ffb-5f18-45ad-9ac3-5b63694612c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:03.639934 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c95ffb-5f18-45ad-9ac3-5b63694612c4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:03.650201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:03.743870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1383/2758] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:17:03 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:17:03.745071 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c604d-fbe4-427d-8075-9bb53b1f7782 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:03.755156 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c604d-fbe4-427d-8075-9bb53b1f7782 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:03.784852 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:03.817187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7da90f-8808-4b3f-a2ff-bac75a14ead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:03.818151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7da90f-8808-4b3f-a2ff-bac75a14ead5 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:03.818643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7da90f-8808-4b3f-a2ff-bac75a14ead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:03.819054 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7da90f-8808-4b3f-a2ff-bac75a14ead5 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:03.820645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7da90f-8808-4b3f-a2ff-bac75a14ead5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:03.823645 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7da90f-8808-4b3f-a2ff-bac75a14ead5 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:03.928993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c95ffb-5f18-45ad-9ac3-5b63694612c4 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:03.930742 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-f6c95ffb-5f18-45ad-9ac3-5b63694612c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:03.943875 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c95ffb-5f18-45ad-9ac3-5b63694612c4 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:03.949216 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:03.952736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:03.955168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:03.955339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] PATH_INFO: `/v3/projects/e024abc8063e488395a29967e633cd61` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:03.956609 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e024abc8063e488395a29967e633cd61)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:03.958426 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1376/2759] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:17:03 2022] POST /identity/v3/users => generated 282 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:17:03.958551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:03.963083 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7da90f-8808-4b3f-a2ff-bac75a14ead5 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:03.972938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7da90f-8808-4b3f-a2ff-bac75a14ead5 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:03.978275 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa720949-7a36-4486-a54e-d47e2bd6f167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:03.978929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa720949-7a36-4486-a54e-d47e2bd6f167 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:03.979214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa720949-7a36-4486-a54e-d47e2bd6f167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:03.979488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa720949-7a36-4486-a54e-d47e2bd6f167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:03.980235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa720949-7a36-4486-a54e-d47e2bd6f167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:03.981818 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa720949-7a36-4486-a54e-d47e2bd6f167 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:03.991900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1377/2760] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:17:03.996308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa720949-7a36-4486-a54e-d47e2bd6f167 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.005757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa720949-7a36-4486-a54e-d47e2bd6f167 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.010367 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e88d9-f818-4c9e-81af-4cd773bf137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.011003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e88d9-f818-4c9e-81af-4cd773bf137c admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.011289 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e88d9-f818-4c9e-81af-4cd773bf137c admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.011566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e88d9-f818-4c9e-81af-4cd773bf137c admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.012274 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e88d9-f818-4c9e-81af-4cd773bf137c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.013977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e88d9-f818-4c9e-81af-4cd773bf137c admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.024618 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1378/2761] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:17:04.029206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e88d9-f818-4c9e-81af-4cd773bf137c admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.038306 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e88d9-f818-4c9e-81af-4cd773bf137c admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.043457 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9299156-1fd3-41e2-9b53-1e854be1cc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.044286 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299156-1fd3-41e2-9b53-1e854be1cc37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.044568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299156-1fd3-41e2-9b53-1e854be1cc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.044950 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299156-1fd3-41e2-9b53-1e854be1cc37 admin admin] PATH_INFO: `/v3/projects/ec16c9b2a6bd4ab385e932015bff9a47/users/fa90505c474742aebc85fec1fb1e3521/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.061915 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e024abc8063e488395a29967e633cd61', 'request_id': 'req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:04.062112 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9299156-1fd3-41e2-9b53-1e854be1cc37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec16c9b2a6bd4ab385e932015bff9a47, user_id=fa90505c474742aebc85fec1fb1e3521, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.064728 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9299156-1fd3-41e2-9b53-1e854be1cc37 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.067764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e024abc8063e488395a29967e633cd61', 'request_id': 'req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:04.075290 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e024abc8063e488395a29967e633cd61', 'request_id': 'req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:04.134748 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1379/2762] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:17:04 2022] PUT /identity/v3/projects/ec16c9b2a6bd4ab385e932015bff9a47/users/fa90505c474742aebc85fec1fb1e3521/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:17:04.144872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985e617-8083-4e7e-867b-d25b00226a10 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.145169 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985e617-8083-4e7e-867b-d25b00226a10 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.145450 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985e617-8083-4e7e-867b-d25b00226a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.145564 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] The token cache is being invalidate because project e024abc8063e488395a29967e633cd61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:17:04.146008 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:04.146398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-6e5b8d4b-78ee-4455-bc3a-e53ef1b5539f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:04.148092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1384/2763] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:17:03 2022] DELETE /identity/v3/projects/e024abc8063e488395a29967e633cd61 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:17:04.154522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654fd74-9288-4c1a-8f98-0a8dbe48491f None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.154839 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654fd74-9288-4c1a-8f98-0a8dbe48491f None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.155197 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654fd74-9288-4c1a-8f98-0a8dbe48491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.180702 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5654fd74-9288-4c1a-8f98-0a8dbe48491f None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:04.181553 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-8985e617-8083-4e7e-867b-d25b00226a10 None None] MFA Rules not processed for user `fa90505c474742aebc85fec1fb1e3521`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:04.182367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5654fd74-9288-4c1a-8f98-0a8dbe48491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:04.184337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8985e617-8083-4e7e-867b-d25b00226a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:04.246821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1385/2764] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:17:04.255386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9a31-f8a3-467b-9ede-14ba7c9e47e6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.255677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9a31-f8a3-467b-9ede-14ba7c9e47e6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.255956 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e9a31-f8a3-467b-9ede-14ba7c9e47e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.256742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1380/2765] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:04.261529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985e617-8083-4e7e-867b-d25b00226a10 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.262788 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-8985e617-8083-4e7e-867b-d25b00226a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:04.276164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985e617-8083-4e7e-867b-d25b00226a10 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.277363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c13e9a31-f8a3-467b-9ede-14ba7c9e47e6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:04.278964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c13e9a31-f8a3-467b-9ede-14ba7c9e47e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:04.280676 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2730a-94a2-4647-8206-25801536a992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.281357 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2730a-94a2-4647-8206-25801536a992 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.281639 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2730a-94a2-4647-8206-25801536a992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.281913 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2730a-94a2-4647-8206-25801536a992 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.282599 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2730a-94a2-4647-8206-25801536a992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.284476 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2730a-94a2-4647-8206-25801536a992 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.291488 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1386/2766] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:17:04.292756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e9a31-f8a3-467b-9ede-14ba7c9e47e6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.302799 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e9a31-f8a3-467b-9ede-14ba7c9e47e6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.307444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9238710-cb1b-4177-b84a-186067b48062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.314981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9238710-cb1b-4177-b84a-186067b48062 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.314981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9238710-cb1b-4177-b84a-186067b48062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.314981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9238710-cb1b-4177-b84a-186067b48062 admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.314981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9238710-cb1b-4177-b84a-186067b48062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:17:04.314981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9238710-cb1b-4177-b84a-186067b48062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.314981 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9238710-cb1b-4177-b84a-186067b48062 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.322636 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1387/2767] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:17:04.326737 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9238710-cb1b-4177-b84a-186067b48062 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.336496 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9238710-cb1b-4177-b84a-186067b48062 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.341101 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb8268-8c91-4c16-97af-510b2d3ec7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.341699 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb8268-8c91-4c16-97af-510b2d3ec7bb admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.343685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb8268-8c91-4c16-97af-510b2d3ec7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.343685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb8268-8c91-4c16-97af-510b2d3ec7bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.343685 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb8268-8c91-4c16-97af-510b2d3ec7bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.344697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb8268-8c91-4c16-97af-510b2d3ec7bb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.369778 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1381/2768] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:17:04 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:17:04.373768 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2730a-94a2-4647-8206-25801536a992 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.415128 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:04.429929 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1388/2769] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:17:04 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:17:04.431001 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb8268-8c91-4c16-97af-510b2d3ec7bb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.434102 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2730a-94a2-4647-8206-25801536a992 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.466628 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:04.495975 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb8268-8c91-4c16-97af-510b2d3ec7bb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.498799 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2f917-4784-4147-aa79-3a452802fc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.499544 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2f917-4784-4147-aa79-3a452802fc50 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.499786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2f917-4784-4147-aa79-3a452802fc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.500066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2f917-4784-4147-aa79-3a452802fc50 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.500758 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2f917-4784-4147-aa79-3a452802fc50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.501544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b44c5-85eb-4e6c-8520-253d639a24b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.502210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b44c5-85eb-4e6c-8520-253d639a24b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.502494 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b44c5-85eb-4e6c-8520-253d639a24b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.502695 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2f917-4784-4147-aa79-3a452802fc50 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.502794 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b44c5-85eb-4e6c-8520-253d639a24b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.503469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b44c5-85eb-4e6c-8520-253d639a24b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.505896 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b44c5-85eb-4e6c-8520-253d639a24b2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.602848 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1382/2771] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:17:04 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:17:04.603061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1389/2771] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:17:04 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:17:04.606612 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2f917-4784-4147-aa79-3a452802fc50 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.606818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b44c5-85eb-4e6c-8520-253d639a24b2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.616928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2f917-4784-4147-aa79-3a452802fc50 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.618022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b44c5-85eb-4e6c-8520-253d639a24b2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.621846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30011deb-2dd7-4e04-abf1-3afac3d0e9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.622494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30011deb-2dd7-4e04-abf1-3afac3d0e9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.622631 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a15351-7bbc-4b21-99c3-3ad1f07bba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.622777 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30011deb-2dd7-4e04-abf1-3afac3d0e9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.623050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30011deb-2dd7-4e04-abf1-3afac3d0e9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.623239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a15351-7bbc-4b21-99c3-3ad1f07bba7d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.623524 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a15351-7bbc-4b21-99c3-3ad1f07bba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.623760 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30011deb-2dd7-4e04-abf1-3afac3d0e9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.623864 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a15351-7bbc-4b21-99c3-3ad1f07bba7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.624537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a15351-7bbc-4b21-99c3-3ad1f07bba7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.625453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30011deb-2dd7-4e04-abf1-3afac3d0e9a3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.626181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a15351-7bbc-4b21-99c3-3ad1f07bba7d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.639160 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1383/2772] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:17:04.639338 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1390/2773] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:17:04.640159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30011deb-2dd7-4e04-abf1-3afac3d0e9a3 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.640304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a15351-7bbc-4b21-99c3-3ad1f07bba7d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.650322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a15351-7bbc-4b21-99c3-3ad1f07bba7d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.650834 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30011deb-2dd7-4e04-abf1-3afac3d0e9a3 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.654901 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a09fb-d254-467d-94a2-c9551dd5e8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.655543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a09fb-d254-467d-94a2-c9551dd5e8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.655687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9217f-36ce-4226-9d6f-b3dbad3635d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.655811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a09fb-d254-467d-94a2-c9551dd5e8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.656092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a09fb-d254-467d-94a2-c9551dd5e8a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.656568 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9217f-36ce-4226-9d6f-b3dbad3635d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.656837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a09fb-d254-467d-94a2-c9551dd5e8a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.656935 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9217f-36ce-4226-9d6f-b3dbad3635d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.657217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9217f-36ce-4226-9d6f-b3dbad3635d3 admin admin] PATH_INFO: `/v3/projects/d42dd35f0929436d9af59e7bdf87aa28/users/5e89c5c7cf974b89990a7e8c244f9867/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.658408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a09fb-d254-467d-94a2-c9551dd5e8a2 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.670000 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1391/2774] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:17:04.675425 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9217f-36ce-4226-9d6f-b3dbad3635d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42dd35f0929436d9af59e7bdf87aa28, user_id=5e89c5c7cf974b89990a7e8c244f9867, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.675567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a09fb-d254-467d-94a2-c9551dd5e8a2 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.675681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9217f-36ce-4226-9d6f-b3dbad3635d3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.683967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a09fb-d254-467d-94a2-c9551dd5e8a2 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.688922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8c13b-c664-4af0-827b-f3e99fd1a3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.689775 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8c13b-c664-4af0-827b-f3e99fd1a3dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.690054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8c13b-c664-4af0-827b-f3e99fd1a3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.690328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8c13b-c664-4af0-827b-f3e99fd1a3dd admin admin] PATH_INFO: `/v3/projects/fc87b58688324722943e6fff1e6ab685/users/acdc746b64d6422ca521175867bde06d/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.705490 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8c13b-c664-4af0-827b-f3e99fd1a3dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc87b58688324722943e6fff1e6ab685, user_id=acdc746b64d6422ca521175867bde06d, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.708269 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae8c13b-c664-4af0-827b-f3e99fd1a3dd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.805796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1384/2775] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:17:04 2022] PUT /identity/v3/projects/d42dd35f0929436d9af59e7bdf87aa28/users/5e89c5c7cf974b89990a7e8c244f9867/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:04.810022 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9217f-36ce-4226-9d6f-b3dbad3635d3 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.820324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1392/2776] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:17:04 2022] PUT /identity/v3/projects/fc87b58688324722943e6fff1e6ab685/users/acdc746b64d6422ca521175867bde06d/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:17:04.830909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd945a-6e1c-430d-89fb-22c49d4fd5e4 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.831258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd945a-6e1c-430d-89fb-22c49d4fd5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.831545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd945a-6e1c-430d-89fb-22c49d4fd5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.861592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-58cd945a-6e1c-430d-89fb-22c49d4fd5e4 None None] MFA Rules not processed for user `acdc746b64d6422ca521175867bde06d`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:04.862863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-58cd945a-6e1c-430d-89fb-22c49d4fd5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:04.873448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9217f-36ce-4226-9d6f-b3dbad3635d3 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.918554 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:04.927102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1393/2777] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:04.931240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932107b-b075-481d-9c48-308003c7ab30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.933556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932107b-b075-481d-9c48-308003c7ab30 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.933556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932107b-b075-481d-9c48-308003c7ab30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.933556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932107b-b075-481d-9c48-308003c7ab30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.933556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932107b-b075-481d-9c48-308003c7ab30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.935300 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932107b-b075-481d-9c48-308003c7ab30 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.939437 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd945a-6e1c-430d-89fb-22c49d4fd5e4 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.940888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-58cd945a-6e1c-430d-89fb-22c49d4fd5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:04.947075 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1385/2778] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:17:04.951415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932107b-b075-481d-9c48-308003c7ab30 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:04.960667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932107b-b075-481d-9c48-308003c7ab30 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:04.965761 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d611d-ed7f-4c07-bc85-5a79abb1e2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:04.966630 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d611d-ed7f-4c07-bc85-5a79abb1e2ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.966912 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d611d-ed7f-4c07-bc85-5a79abb1e2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.967193 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d611d-ed7f-4c07-bc85-5a79abb1e2ad admin admin] PATH_INFO: `/v3/projects/d42dd35f0929436d9af59e7bdf87aa28/users/5e89c5c7cf974b89990a7e8c244f9867/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:04.969159 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d611d-ed7f-4c07-bc85-5a79abb1e2ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42dd35f0929436d9af59e7bdf87aa28, user_id=5e89c5c7cf974b89990a7e8c244f9867, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:04.971786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d611d-ed7f-4c07-bc85-5a79abb1e2ad admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:04.982921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1386/2779] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:17:04 2022] PUT /identity/v3/projects/d42dd35f0929436d9af59e7bdf87aa28/users/5e89c5c7cf974b89990a7e8c244f9867/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:17:04.993649 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282b1f-48b8-46e8-bcbb-6bed34358684 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:04.993940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282b1f-48b8-46e8-bcbb-6bed34358684 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:04.994220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282b1f-48b8-46e8-bcbb-6bed34358684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:05.003059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd945a-6e1c-430d-89fb-22c49d4fd5e4 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:05.020660 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4d282b1f-48b8-46e8-bcbb-6bed34358684 None None] MFA Rules not processed for user `5e89c5c7cf974b89990a7e8c244f9867`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:05.021351 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:05.021885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4d282b1f-48b8-46e8-bcbb-6bed34358684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:05.052383 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cff15-c3e1-4996-b3db-231454e841b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5b712bd7d8d4930a3bfc99581be8cb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:05.053092 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cff15-c3e1-4996-b3db-231454e841b5 service glance] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:05.053378 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cff15-c3e1-4996-b3db-231454e841b5 service glance] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:05.053655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cff15-c3e1-4996-b3db-231454e841b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:05.091383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1387/2780] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:17:05.104439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282b1f-48b8-46e8-bcbb-6bed34358684 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:05.106384 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4d282b1f-48b8-46e8-bcbb-6bed34358684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:05.115515 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cff15-c3e1-4996-b3db-231454e841b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:05.117045 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cff15-c3e1-4996-b3db-231454e841b5 service glance] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:05.126164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1394/2781] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:17:05.173336 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282b1f-48b8-46e8-bcbb-6bed34358684 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:05.177543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a197b0-f5cd-41bc-b841-7c9935bc11f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:05.178269 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a197b0-f5cd-41bc-b841-7c9935bc11f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:05.178629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a197b0-f5cd-41bc-b841-7c9935bc11f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:05.179214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a197b0-f5cd-41bc-b841-7c9935bc11f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:05.189519 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a197b0-f5cd-41bc-b841-7c9935bc11f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:05.191255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a197b0-f5cd-41bc-b841-7c9935bc11f9 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:05.202041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1388/2782] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 16:17:06.066028 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cff15-c3e1-4996-b3db-231454e841b5 service glance] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:06.129677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cff15-c3e1-4996-b3db-231454e841b5 service glance] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:06.134874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:06.135764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:06.136057 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:06.136335 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] PATH_INFO: `/v3/users/fa90505c474742aebc85fec1fb1e3521` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:06.137568 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa90505c474742aebc85fec1fb1e3521)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:06.139542 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:06.317853 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa90505c474742aebc85fec1fb1e3521'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:06.325835 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa90505c474742aebc85fec1fb1e3521'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:06.326489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa90505c474742aebc85fec1fb1e3521'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:06.327219 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa90505c474742aebc85fec1fb1e3521'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:06.339578 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa90505c474742aebc85fec1fb1e3521'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:06.343178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:06.414873 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1395/2783] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:17:06 2022] DELETE /identity/v3/users/fa90505c474742aebc85fec1fb1e3521 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:06.419442 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050524a9-edaf-497c-954c-21399c2e76ee None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:06.419958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050524a9-edaf-497c-954c-21399c2e76ee None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:06.420069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050524a9-edaf-497c-954c-21399c2e76ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:06.438751 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-050524a9-edaf-497c-954c-21399c2e76ee None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:06.439956 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-050524a9-edaf-497c-954c-21399c2e76ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:06.499744 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:06.507905 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1389/2784] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:06.517519 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:06.518909 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:06.579387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313c753-1d4b-4ab9-9430-cf049c30897a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:06.583024 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2506c-e300-4863-aa31-b24d193e24c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:06.583678 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2506c-e300-4863-aa31-b24d193e24c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:06.583963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2506c-e300-4863-aa31-b24d193e24c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:06.584241 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2506c-e300-4863-aa31-b24d193e24c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:06.594551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2506c-e300-4863-aa31-b24d193e24c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:06.596029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2506c-e300-4863-aa31-b24d193e24c3 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:06.606165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1396/2785] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:07.037276 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050524a9-edaf-497c-954c-21399c2e76ee None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:07.039061 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-050524a9-edaf-497c-954c-21399c2e76ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:07.051293 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050524a9-edaf-497c-954c-21399c2e76ee None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:07.056589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:07.057593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:07.057923 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:07.058170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] PATH_INFO: `/v3/projects/ec16c9b2a6bd4ab385e932015bff9a47` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:07.059489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec16c9b2a6bd4ab385e932015bff9a47)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:07.061864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:07.167371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec16c9b2a6bd4ab385e932015bff9a47', 'request_id': 'req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:07.173427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec16c9b2a6bd4ab385e932015bff9a47', 'request_id': 'req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:07.236972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec16c9b2a6bd4ab385e932015bff9a47', 'request_id': 'req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:07.248204 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] The token cache is being invalidate because project ec16c9b2a6bd4ab385e932015bff9a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:17:07.248487 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:07.249104 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:07.250614 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1390/2786] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:17:07 2022] DELETE /identity/v3/projects/ec16c9b2a6bd4ab385e932015bff9a47 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:07.254562 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2506c-e300-4863-aa31-b24d193e24c3 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:07.264694 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2506c-e300-4863-aa31-b24d193e24c3 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:07.325541 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:07.326345 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:07.326630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:07.326910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] PATH_INFO: `/v3/users/5e89c5c7cf974b89990a7e8c244f9867` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:07.328102 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e89c5c7cf974b89990a7e8c244f9867)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:07.330049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:07.525016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e89c5c7cf974b89990a7e8c244f9867'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:07.531964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e89c5c7cf974b89990a7e8c244f9867'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:07.532677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e89c5c7cf974b89990a7e8c244f9867'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:07.533308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e89c5c7cf974b89990a7e8c244f9867'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:07.549458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-868301b1-d4b8-4f67-a0ee-225332a0574f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e89c5c7cf974b89990a7e8c244f9867'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:07.636949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1397/2787] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:17:07 2022] DELETE /identity/v3/users/5e89c5c7cf974b89990a7e8c244f9867 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:08.072885 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:08.077731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:08.089373 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc34b9-554d-4b88-96be-0cc5ef3da6e6 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:08.149477 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:08.150239 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:08.150521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:08.150796 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] PATH_INFO: `/v3/projects/d42dd35f0929436d9af59e7bdf87aa28` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:08.151990 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d42dd35f0929436d9af59e7bdf87aa28)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:08.153925 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:08.249036 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd42dd35f0929436d9af59e7bdf87aa28', 'request_id': 'req-82beac4b-0dc1-4ee7-a620-92de126c5e1e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:08.255131 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd42dd35f0929436d9af59e7bdf87aa28', 'request_id': 'req-82beac4b-0dc1-4ee7-a620-92de126c5e1e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:08.317214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd42dd35f0929436d9af59e7bdf87aa28', 'request_id': 'req-82beac4b-0dc1-4ee7-a620-92de126c5e1e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:08.329344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] The token cache is being invalidate because project d42dd35f0929436d9af59e7bdf87aa28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:17:08.329344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:08.329344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-82beac4b-0dc1-4ee7-a620-92de126c5e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:08.330968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1391/2788] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:17:08 2022] DELETE /identity/v3/projects/d42dd35f0929436d9af59e7bdf87aa28 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:08.336818 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8eec0-691c-4c4b-9cfa-ccb12756fa18 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:08.337771 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8eec0-691c-4c4b-9cfa-ccb12756fa18 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:08.338293 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8eec0-691c-4c4b-9cfa-ccb12756fa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:08.360245 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-57e8eec0-691c-4c4b-9cfa-ccb12756fa18 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:08.361506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-57e8eec0-691c-4c4b-9cfa-ccb12756fa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:08.374297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:08.421800 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1398/2789] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:08.430999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936930b8-8d4e-49a9-8c0e-59d26297fc57 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:08.431944 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936930b8-8d4e-49a9-8c0e-59d26297fc57 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:08.432225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936930b8-8d4e-49a9-8c0e-59d26297fc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:08.450255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-936930b8-8d4e-49a9-8c0e-59d26297fc57 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:08.451545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-936930b8-8d4e-49a9-8c0e-59d26297fc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:08.459838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1392/2790] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:08.464586 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8eec0-691c-4c4b-9cfa-ccb12756fa18 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:08.475837 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8eec0-691c-4c4b-9cfa-ccb12756fa18 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:08.481095 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c414-ce72-4b5d-a1ab-4a3998dfff8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:08.481756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c414-ce72-4b5d-a1ab-4a3998dfff8d admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:08.482041 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c414-ce72-4b5d-a1ab-4a3998dfff8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:08.482317 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c414-ce72-4b5d-a1ab-4a3998dfff8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:08.483043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2c414-ce72-4b5d-a1ab-4a3998dfff8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:17:08.483380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2c414-ce72-4b5d-a1ab-4a3998dfff8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:08.485270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2c414-ce72-4b5d-a1ab-4a3998dfff8d admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:08.497446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1399/2791] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:17:08.501824 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936930b8-8d4e-49a9-8c0e-59d26297fc57 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:08.511486 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936930b8-8d4e-49a9-8c0e-59d26297fc57 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:08.516066 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a84b5-b7cb-43ae-801d-fc8d4d484dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:08.516705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a84b5-b7cb-43ae-801d-fc8d4d484dee admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:08.517020 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a84b5-b7cb-43ae-801d-fc8d4d484dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:08.517296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a84b5-b7cb-43ae-801d-fc8d4d484dee admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:08.517980 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a84b5-b7cb-43ae-801d-fc8d4d484dee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:08.519846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a84b5-b7cb-43ae-801d-fc8d4d484dee admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:08.522979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:08.616421 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1393/2792] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:17:08 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 16:17:08.621105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c414-ce72-4b5d-a1ab-4a3998dfff8d admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:08.631058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c414-ce72-4b5d-a1ab-4a3998dfff8d admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:08.688906 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129a0b6-1e8a-4d01-abc2-7adabced8adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:08.689580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129a0b6-1e8a-4d01-abc2-7adabced8adb admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:08.689894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129a0b6-1e8a-4d01-abc2-7adabced8adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:08.690172 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129a0b6-1e8a-4d01-abc2-7adabced8adb admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:08.690902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129a0b6-1e8a-4d01-abc2-7adabced8adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:08.692856 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129a0b6-1e8a-4d01-abc2-7adabced8adb admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:08.785992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1400/2793] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:17:08 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:17:08.790078 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a84b5-b7cb-43ae-801d-fc8d4d484dee admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:08.800316 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a84b5-b7cb-43ae-801d-fc8d4d484dee admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:08.805051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe43a9a-05c3-429e-bb7d-7771b11c4a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:08.805710 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe43a9a-05c3-429e-bb7d-7771b11c4a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:08.805994 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe43a9a-05c3-429e-bb7d-7771b11c4a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:08.806782 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe43a9a-05c3-429e-bb7d-7771b11c4a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:08.807509 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe43a9a-05c3-429e-bb7d-7771b11c4a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:08.809140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe43a9a-05c3-429e-bb7d-7771b11c4a5a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:08.821062 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1394/2794] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:17:08.825697 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129a0b6-1e8a-4d01-abc2-7adabced8adb admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:08.835719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129a0b6-1e8a-4d01-abc2-7adabced8adb admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:08.840332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26d127-cf49-4148-8d6a-5fede9d9e389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:08.841011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d127-cf49-4148-8d6a-5fede9d9e389 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:08.841297 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d127-cf49-4148-8d6a-5fede9d9e389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:08.841575 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26d127-cf49-4148-8d6a-5fede9d9e389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:08.842298 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26d127-cf49-4148-8d6a-5fede9d9e389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:08.843894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26d127-cf49-4148-8d6a-5fede9d9e389 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:08.854236 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1401/2795] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:17:08.858058 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe43a9a-05c3-429e-bb7d-7771b11c4a5a admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:08.869604 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe43a9a-05c3-429e-bb7d-7771b11c4a5a admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:08.874801 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f94f6-22f9-4277-b14f-3f22282ef242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:08.875694 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f94f6-22f9-4277-b14f-3f22282ef242 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:08.875979 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f94f6-22f9-4277-b14f-3f22282ef242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:08.876285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f94f6-22f9-4277-b14f-3f22282ef242 admin admin] PATH_INFO: `/v3/projects/2714e023d77f49cbbb94dd36776e6896/users/31affa535d97471ebef42fdeaf3458d3/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:08.890970 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f94f6-22f9-4277-b14f-3f22282ef242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2714e023d77f49cbbb94dd36776e6896, user_id=31affa535d97471ebef42fdeaf3458d3, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:08.893625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f94f6-22f9-4277-b14f-3f22282ef242 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:08.985393 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1395/2796] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:17:08 2022] PUT /identity/v3/projects/2714e023d77f49cbbb94dd36776e6896/users/31affa535d97471ebef42fdeaf3458d3/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 16:17:08.996307 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304c84-aed9-40e9-86c0-13988ee530ec None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:08.996652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304c84-aed9-40e9-86c0-13988ee530ec None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:08.996963 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304c84-aed9-40e9-86c0-13988ee530ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:09.023577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ce304c84-aed9-40e9-86c0-13988ee530ec None None] MFA Rules not processed for user `31affa535d97471ebef42fdeaf3458d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:09.024895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ce304c84-aed9-40e9-86c0-13988ee530ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:09.092274 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1402/2797] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:09.105545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f94f6-22f9-4277-b14f-3f22282ef242 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:09.107466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-605f94f6-22f9-4277-b14f-3f22282ef242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:09.163636 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f94f6-22f9-4277-b14f-3f22282ef242 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:09.167959 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03386af-1e2e-4c64-9eff-097b0b6f8379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:09.168621 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03386af-1e2e-4c64-9eff-097b0b6f8379 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:09.168937 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03386af-1e2e-4c64-9eff-097b0b6f8379 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:09.169217 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03386af-1e2e-4c64-9eff-097b0b6f8379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:09.178383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03386af-1e2e-4c64-9eff-097b0b6f8379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:09.179814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03386af-1e2e-4c64-9eff-097b0b6f8379 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:09.189281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1396/2798] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:13.174324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304c84-aed9-40e9-86c0-13988ee530ec None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:13.189778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:13.245846 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304c84-aed9-40e9-86c0-13988ee530ec None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:13.250329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:13.251224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:13.251507 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:13.251787 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] PATH_INFO: `/v3/users/31affa535d97471ebef42fdeaf3458d3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:13.253037 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31affa535d97471ebef42fdeaf3458d3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:13.254961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:13.541545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31affa535d97471ebef42fdeaf3458d3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:13.548602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31affa535d97471ebef42fdeaf3458d3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:13.549289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31affa535d97471ebef42fdeaf3458d3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:13.550254 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31affa535d97471ebef42fdeaf3458d3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:13.562304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31affa535d97471ebef42fdeaf3458d3'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:13.625149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1403/2799] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:17:13 2022] DELETE /identity/v3/users/31affa535d97471ebef42fdeaf3458d3 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:13.629924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf7882-0040-4360-809f-c2dd1a207d74 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:13.630430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf7882-0040-4360-809f-c2dd1a207d74 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:13.630536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf7882-0040-4360-809f-c2dd1a207d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:13.648130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-2fcf7882-0040-4360-809f-c2dd1a207d74 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:13.656518 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2fcf7882-0040-4360-809f-c2dd1a207d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:13.707991 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1397/2800] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:13.718407 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:13.719834 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:13.779268 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bae84e-6160-4333-8ede-fc8685abc723 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:13.782806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a58d2-cc35-4982-a2a3-6328165386b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:13.783556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a58d2-cc35-4982-a2a3-6328165386b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:13.784036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a58d2-cc35-4982-a2a3-6328165386b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:13.784504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a58d2-cc35-4982-a2a3-6328165386b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:13.798168 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a58d2-cc35-4982-a2a3-6328165386b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:13.799530 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a58d2-cc35-4982-a2a3-6328165386b5 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:13.810792 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1404/2801] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:14.255661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf7882-0040-4360-809f-c2dd1a207d74 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:14.258370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2fcf7882-0040-4360-809f-c2dd1a207d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:14.270594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf7882-0040-4360-809f-c2dd1a207d74 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:14.275415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:14.276359 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:14.276716 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:14.277097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] PATH_INFO: `/v3/projects/2714e023d77f49cbbb94dd36776e6896` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:14.278409 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2714e023d77f49cbbb94dd36776e6896)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:14.280394 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:14.413250 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:14.414598 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2714e023d77f49cbbb94dd36776e6896', 'request_id': 'req-dcdd3553-3365-428d-910e-ee76567a7095'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:14.421406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2714e023d77f49cbbb94dd36776e6896', 'request_id': 'req-dcdd3553-3365-428d-910e-ee76567a7095'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:14.488194 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2714e023d77f49cbbb94dd36776e6896', 'request_id': 'req-dcdd3553-3365-428d-910e-ee76567a7095'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:14.499168 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] The token cache is being invalidate because project 2714e023d77f49cbbb94dd36776e6896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:17:14.505657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:14.505657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-dcdd3553-3365-428d-910e-ee76567a7095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:14.505657 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1398/2802] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:17:14 2022] DELETE /identity/v3/projects/2714e023d77f49cbbb94dd36776e6896 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:14.512018 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0545d-9d74-4bd7-bb93-9daf894ea2e0 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:14.512698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0545d-9d74-4bd7-bb93-9daf894ea2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:14.512698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0545d-9d74-4bd7-bb93-9daf894ea2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:14.531202 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-68c0545d-9d74-4bd7-bb93-9daf894ea2e0 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:14.532392 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-68c0545d-9d74-4bd7-bb93-9daf894ea2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:14.564602 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:14.595797 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1405/2803] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:14.604142 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d361-f042-4847-806e-7370339c7ef2 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:14.604429 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d361-f042-4847-806e-7370339c7ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:14.604702 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d361-f042-4847-806e-7370339c7ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:14.622977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-b921d361-f042-4847-806e-7370339c7ef2 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:14.629655 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-b921d361-f042-4847-806e-7370339c7ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:14.636828 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1399/2804] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:14.641059 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0545d-9d74-4bd7-bb93-9daf894ea2e0 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:14.651583 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0545d-9d74-4bd7-bb93-9daf894ea2e0 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:14.656181 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df7741-f14f-4449-9068-c99fbc6a7cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:14.656864 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df7741-f14f-4449-9068-c99fbc6a7cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:14.657149 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df7741-f14f-4449-9068-c99fbc6a7cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:14.657426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df7741-f14f-4449-9068-c99fbc6a7cdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:14.658152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df7741-f14f-4449-9068-c99fbc6a7cdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:17:14.658456 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df7741-f14f-4449-9068-c99fbc6a7cdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:14.660044 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df7741-f14f-4449-9068-c99fbc6a7cdc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:14.671884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1406/2805] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:17:14.676225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921d361-f042-4847-806e-7370339c7ef2 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:14.685435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921d361-f042-4847-806e-7370339c7ef2 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:14.692113 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171ba30-4155-4bb9-bd17-44cf0c0b6c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:14.693822 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171ba30-4155-4bb9-bd17-44cf0c0b6c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:14.694121 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171ba30-4155-4bb9-bd17-44cf0c0b6c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:14.694523 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171ba30-4155-4bb9-bd17-44cf0c0b6c06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:14.695928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171ba30-4155-4bb9-bd17-44cf0c0b6c06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:14.698607 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171ba30-4155-4bb9-bd17-44cf0c0b6c06 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:14.827489 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1400/2806] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:17:14 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:17:14.831649 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df7741-f14f-4449-9068-c99fbc6a7cdc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:14.844208 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df7741-f14f-4449-9068-c99fbc6a7cdc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:14.902311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d48f3f-35b2-4d2c-aa86-464d58670062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:14.902910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d48f3f-35b2-4d2c-aa86-464d58670062 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:14.903224 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d48f3f-35b2-4d2c-aa86-464d58670062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:14.903501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d48f3f-35b2-4d2c-aa86-464d58670062 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:14.904195 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d48f3f-35b2-4d2c-aa86-464d58670062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:14.906155 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d48f3f-35b2-4d2c-aa86-464d58670062 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:15.010706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1407/2807] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:17:14 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:17:15.014654 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171ba30-4155-4bb9-bd17-44cf0c0b6c06 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:15.025004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171ba30-4155-4bb9-bd17-44cf0c0b6c06 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:15.030467 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3f664-0882-46a6-86b5-41510f99bc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:15.031155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3f664-0882-46a6-86b5-41510f99bc20 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:15.031439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3f664-0882-46a6-86b5-41510f99bc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:15.031737 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3f664-0882-46a6-86b5-41510f99bc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:15.032462 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3f664-0882-46a6-86b5-41510f99bc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:15.034086 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3f664-0882-46a6-86b5-41510f99bc20 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:15.045327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1401/2808] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:17:15.049217 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d48f3f-35b2-4d2c-aa86-464d58670062 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:15.059209 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d48f3f-35b2-4d2c-aa86-464d58670062 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:15.064087 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdc7d6-4cca-4e27-ab8a-39b897893050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:15.064809 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdc7d6-4cca-4e27-ab8a-39b897893050 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:15.065793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdc7d6-4cca-4e27-ab8a-39b897893050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:15.065793 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdc7d6-4cca-4e27-ab8a-39b897893050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:15.066424 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdc7d6-4cca-4e27-ab8a-39b897893050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:15.068624 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdc7d6-4cca-4e27-ab8a-39b897893050 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:15.080015 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1408/2809] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:17:15.084099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3f664-0882-46a6-86b5-41510f99bc20 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:15.094680 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3f664-0882-46a6-86b5-41510f99bc20 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:15.099170 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa2840-3d1c-40bf-810d-d9f2270b4751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:15.100024 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa2840-3d1c-40bf-810d-d9f2270b4751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:15.100308 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa2840-3d1c-40bf-810d-d9f2270b4751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:15.100579 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa2840-3d1c-40bf-810d-d9f2270b4751 admin admin] PATH_INFO: `/v3/projects/aa9a393ca8c143d9b0a129d3c0c55955/users/598e5dd6bfc04c0bb9af94c01140e28b/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:15.114779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa2840-3d1c-40bf-810d-d9f2270b4751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9a393ca8c143d9b0a129d3c0c55955, user_id=598e5dd6bfc04c0bb9af94c01140e28b, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:15.117438 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa2840-3d1c-40bf-810d-d9f2270b4751 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:15.230103 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1402/2810] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:17:15 2022] PUT /identity/v3/projects/aa9a393ca8c143d9b0a129d3c0c55955/users/598e5dd6bfc04c0bb9af94c01140e28b/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:15.241170 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be581-18e9-4cda-9330-66db1143dd82 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:15.241458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be581-18e9-4cda-9330-66db1143dd82 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:15.241735 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629be581-18e9-4cda-9330-66db1143dd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:15.269958 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-629be581-18e9-4cda-9330-66db1143dd82 None None] MFA Rules not processed for user `598e5dd6bfc04c0bb9af94c01140e28b`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:15.271171 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-629be581-18e9-4cda-9330-66db1143dd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:15.335029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1409/2811] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:15.346241 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa2840-3d1c-40bf-810d-d9f2270b4751 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:15.347920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-45fa2840-3d1c-40bf-810d-d9f2270b4751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:15.406819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa2840-3d1c-40bf-810d-d9f2270b4751 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:15.410386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec20771-f702-46dd-a5b1-17511a0beb05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:15.411029 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec20771-f702-46dd-a5b1-17511a0beb05 service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:15.411313 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec20771-f702-46dd-a5b1-17511a0beb05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:15.411588 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec20771-f702-46dd-a5b1-17511a0beb05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:15.419866 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec20771-f702-46dd-a5b1-17511a0beb05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:15.421314 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec20771-f702-46dd-a5b1-17511a0beb05 service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:15.429975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1403/2812] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:18.490503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629be581-18e9-4cda-9330-66db1143dd82 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:18.525680 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:18.555020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629be581-18e9-4cda-9330-66db1143dd82 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:18.565486 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:18.566473 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:18.566770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:18.567029 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] PATH_INFO: `/v3/users/598e5dd6bfc04c0bb9af94c01140e28b` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:18.568201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] RBAC: Authorizing `identity:delete_user(user_id=598e5dd6bfc04c0bb9af94c01140e28b)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:18.570147 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:18.723736 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '598e5dd6bfc04c0bb9af94c01140e28b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:18.731258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '598e5dd6bfc04c0bb9af94c01140e28b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:18.731967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '598e5dd6bfc04c0bb9af94c01140e28b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:18.732657 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '598e5dd6bfc04c0bb9af94c01140e28b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:18.745716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '598e5dd6bfc04c0bb9af94c01140e28b'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:18.804398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1410/2813] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:17:18 2022] DELETE /identity/v3/users/598e5dd6bfc04c0bb9af94c01140e28b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:18.809631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eccf5-703c-4845-af07-194559d3f178 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:18.810126 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eccf5-703c-4845-af07-194559d3f178 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:18.810235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eccf5-703c-4845-af07-194559d3f178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:18.829179 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4e7eccf5-703c-4845-af07-194559d3f178 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:18.830380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4e7eccf5-703c-4845-af07-194559d3f178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:18.888862 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1404/2814] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:18.898332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:18.899726 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:18.956271 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d5715-ae2f-4478-852a-4b1ccd76532e admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:18.959851 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f18cd-5d90-4149-a529-1fc6f3e6b1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:18.960482 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f18cd-5d90-4149-a529-1fc6f3e6b1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:18.960778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f18cd-5d90-4149-a529-1fc6f3e6b1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:18.961068 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f18cd-5d90-4149-a529-1fc6f3e6b1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:18.969928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f18cd-5d90-4149-a529-1fc6f3e6b1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:18.971358 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f18cd-5d90-4149-a529-1fc6f3e6b1e3 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:18.980663 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1411/2815] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:19.381877 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7eccf5-703c-4845-af07-194559d3f178 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:19.383661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4e7eccf5-703c-4845-af07-194559d3f178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:19.398246 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7eccf5-703c-4845-af07-194559d3f178 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:19.404398 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:19.405401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:19.408805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:19.408805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] PATH_INFO: `/v3/projects/aa9a393ca8c143d9b0a129d3c0c55955` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:19.408805 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9a393ca8c143d9b0a129d3c0c55955)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:19.409840 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:19.425586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:19.528571 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9a393ca8c143d9b0a129d3c0c55955', 'request_id': 'req-523615d9-e53c-43ab-83fd-2e36ad5f2062'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:19.534643 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9a393ca8c143d9b0a129d3c0c55955', 'request_id': 'req-523615d9-e53c-43ab-83fd-2e36ad5f2062'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:19.637245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9a393ca8c143d9b0a129d3c0c55955', 'request_id': 'req-523615d9-e53c-43ab-83fd-2e36ad5f2062'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:19.647714 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] The token cache is being invalidate because project aa9a393ca8c143d9b0a129d3c0c55955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:17:19.647999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:19.648619 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-523615d9-e53c-43ab-83fd-2e36ad5f2062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:19.650182 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1405/2816] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:17:19 2022] DELETE /identity/v3/projects/aa9a393ca8c143d9b0a129d3c0c55955 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:19.655522 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48931-aeb5-4acf-9d07-e5780153de54 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:19.655988 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48931-aeb5-4acf-9d07-e5780153de54 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:19.656093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48931-aeb5-4acf-9d07-e5780153de54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:19.673931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c5a48931-aeb5-4acf-9d07-e5780153de54 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:19.675107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c5a48931-aeb5-4acf-9d07-e5780153de54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:19.721638 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:19.733634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1412/2817] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:19.741310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1059cf-41a3-4f31-9381-c6a573d6a4e2 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:19.741606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1059cf-41a3-4f31-9381-c6a573d6a4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:19.741884 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1059cf-41a3-4f31-9381-c6a573d6a4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:19.760732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-da1059cf-41a3-4f31-9381-c6a573d6a4e2 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:19.761952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-da1059cf-41a3-4f31-9381-c6a573d6a4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:19.769793 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1406/2818] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:19.773931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a48931-aeb5-4acf-9d07-e5780153de54 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:19.783681 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a48931-aeb5-4acf-9d07-e5780153de54 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:19.788588 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95736f96-1243-4f43-9715-cbfa099f5e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:19.789266 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95736f96-1243-4f43-9715-cbfa099f5e9a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:19.789594 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95736f96-1243-4f43-9715-cbfa099f5e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:19.789863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95736f96-1243-4f43-9715-cbfa099f5e9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:19.790622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95736f96-1243-4f43-9715-cbfa099f5e9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:17:19.790922 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95736f96-1243-4f43-9715-cbfa099f5e9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:19.793054 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95736f96-1243-4f43-9715-cbfa099f5e9a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:19.806848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1413/2819] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:17:19.811625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1059cf-41a3-4f31-9381-c6a573d6a4e2 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:19.822360 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1059cf-41a3-4f31-9381-c6a573d6a4e2 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:19.829955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bd840-058f-4a7b-bea1-1bbd247bcdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:19.830921 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bd840-058f-4a7b-bea1-1bbd247bcdce admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:19.831341 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bd840-058f-4a7b-bea1-1bbd247bcdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:19.831759 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bd840-058f-4a7b-bea1-1bbd247bcdce admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:19.832872 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bd840-058f-4a7b-bea1-1bbd247bcdce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:19.835911 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bd840-058f-4a7b-bea1-1bbd247bcdce admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:19.923349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1407/2820] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:17:19 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:17:19.929009 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95736f96-1243-4f43-9715-cbfa099f5e9a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:19.938079 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95736f96-1243-4f43-9715-cbfa099f5e9a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:19.993375 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48690cb9-241a-49b6-bc9b-f48add4dcc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:19.994036 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48690cb9-241a-49b6-bc9b-f48add4dcc24 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:19.994321 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48690cb9-241a-49b6-bc9b-f48add4dcc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:19.995109 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48690cb9-241a-49b6-bc9b-f48add4dcc24 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:19.995803 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48690cb9-241a-49b6-bc9b-f48add4dcc24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:19.997753 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48690cb9-241a-49b6-bc9b-f48add4dcc24 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:20.132584 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1414/2821] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:17:19 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:17:20.136802 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bd840-058f-4a7b-bea1-1bbd247bcdce admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:20.147957 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bd840-058f-4a7b-bea1-1bbd247bcdce admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:20.152958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4983ecc-bfe7-474a-bdad-5fd09cd0abeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:20.153642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4983ecc-bfe7-474a-bdad-5fd09cd0abeb admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:20.153974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4983ecc-bfe7-474a-bdad-5fd09cd0abeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:20.154248 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4983ecc-bfe7-474a-bdad-5fd09cd0abeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:20.154975 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4983ecc-bfe7-474a-bdad-5fd09cd0abeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:20.156551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4983ecc-bfe7-474a-bdad-5fd09cd0abeb admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:20.167772 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1408/2822] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:17:20.171943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48690cb9-241a-49b6-bc9b-f48add4dcc24 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:20.181967 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48690cb9-241a-49b6-bc9b-f48add4dcc24 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:20.187131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c55a2-918b-418f-9fc7-2816ba09751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:20.187777 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c55a2-918b-418f-9fc7-2816ba09751a admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:20.188076 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c55a2-918b-418f-9fc7-2816ba09751a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:20.188334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c55a2-918b-418f-9fc7-2816ba09751a admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:20.189110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c55a2-918b-418f-9fc7-2816ba09751a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:20.190740 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c55a2-918b-418f-9fc7-2816ba09751a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:20.200917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1415/2823] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:17:20.204928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4983ecc-bfe7-474a-bdad-5fd09cd0abeb admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:20.214625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4983ecc-bfe7-474a-bdad-5fd09cd0abeb admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:20.219021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f78b0-5ad7-4bd3-a11d-c1e72695aec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:20.219882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f78b0-5ad7-4bd3-a11d-c1e72695aec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:20.220164 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f78b0-5ad7-4bd3-a11d-c1e72695aec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:20.220437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f78b0-5ad7-4bd3-a11d-c1e72695aec3 admin admin] PATH_INFO: `/v3/projects/74d513d7cde3430f880b5dd5641d5356/users/0bb4d27ce2104d1086ef35180812b09e/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:20.236096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f78b0-5ad7-4bd3-a11d-c1e72695aec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d513d7cde3430f880b5dd5641d5356, user_id=0bb4d27ce2104d1086ef35180812b09e, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:20.238846 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f78b0-5ad7-4bd3-a11d-c1e72695aec3 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:20.334296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1409/2824] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:17:20 2022] PUT /identity/v3/projects/74d513d7cde3430f880b5dd5641d5356/users/0bb4d27ce2104d1086ef35180812b09e/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:20.344512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1bfdc-51ec-4e0f-adc3-085701b18d0b None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:20.346415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1bfdc-51ec-4e0f-adc3-085701b18d0b None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:20.346415 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1bfdc-51ec-4e0f-adc3-085701b18d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:20.372724 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-90e1bfdc-51ec-4e0f-adc3-085701b18d0b None None] MFA Rules not processed for user `0bb4d27ce2104d1086ef35180812b09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:20.373970 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-90e1bfdc-51ec-4e0f-adc3-085701b18d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:20.438462 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1416/2825] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:20.450700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f78b0-5ad7-4bd3-a11d-c1e72695aec3 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:20.452448 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-523f78b0-5ad7-4bd3-a11d-c1e72695aec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:20.509120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f78b0-5ad7-4bd3-a11d-c1e72695aec3 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:20.512717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec626a4d-d405-4a63-9f70-c4006ba758ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:20.513379 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626a4d-d405-4a63-9f70-c4006ba758ca service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:20.513669 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626a4d-d405-4a63-9f70-c4006ba758ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:20.513974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec626a4d-d405-4a63-9f70-c4006ba758ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:20.522123 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec626a4d-d405-4a63-9f70-c4006ba758ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:20.523540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec626a4d-d405-4a63-9f70-c4006ba758ca service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:20.532786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1410/2826] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:22.233082 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe4edb-d042-40b9-a0a6-72fbdfea5712 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:22.233577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe4edb-d042-40b9-a0a6-72fbdfea5712 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:22.233577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe4edb-d042-40b9-a0a6-72fbdfea5712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:22.252883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-3ffe4edb-d042-40b9-a0a6-72fbdfea5712 None None] MFA Rules not processed for user `0bb4d27ce2104d1086ef35180812b09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:22.254183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3ffe4edb-d042-40b9-a0a6-72fbdfea5712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:22.261031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1417/2827] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:30.528918 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:32.246131 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:32.340363 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec626a4d-d405-4a63-9f70-c4006ba758ca service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:32.341792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-ec626a4d-d405-4a63-9f70-c4006ba758ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:32.405386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec626a4d-d405-4a63-9f70-c4006ba758ca service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:32.410018 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:32.410874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:32.411158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:32.411435 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] PATH_INFO: `/v3/users/acdc746b64d6422ca521175867bde06d` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:32.412617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acdc746b64d6422ca521175867bde06d)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:32.414566 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:32.626814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acdc746b64d6422ca521175867bde06d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:32.633609 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acdc746b64d6422ca521175867bde06d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:32.634284 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acdc746b64d6422ca521175867bde06d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:32.646819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acdc746b64d6422ca521175867bde06d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:32.647441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acdc746b64d6422ca521175867bde06d'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:32.754429 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1411/2828] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:17:32 2022] DELETE /identity/v3/users/acdc746b64d6422ca521175867bde06d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:32.758848 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9949435-b976-4122-a399-413dbcaf09ea None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:32.759136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9949435-b976-4122-a399-413dbcaf09ea None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:32.759414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9949435-b976-4122-a399-413dbcaf09ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:32.777093 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-e9949435-b976-4122-a399-413dbcaf09ea None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:32.778286 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e9949435-b976-4122-a399-413dbcaf09ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:32.837363 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1418/2829] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:32.846886 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:32.848244 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:32.909227 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7fbd-b0c2-4949-b949-712b4da41c75 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:32.914632 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f2e1b-d939-4c39-922a-01d7e246deb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:32.915260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2e1b-d939-4c39-922a-01d7e246deb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:32.915542 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2e1b-d939-4c39-922a-01d7e246deb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:32.915814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2e1b-d939-4c39-922a-01d7e246deb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:32.925717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f2e1b-d939-4c39-922a-01d7e246deb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:32.927140 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f2e1b-d939-4c39-922a-01d7e246deb6 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:32.937625 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1412/2830] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:33.431677 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9949435-b976-4122-a399-413dbcaf09ea None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:33.433710 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-e9949435-b976-4122-a399-413dbcaf09ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:33.445789 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9949435-b976-4122-a399-413dbcaf09ea None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:33.450334 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:33.451194 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:33.451498 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:33.451773 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] PATH_INFO: `/v3/projects/fc87b58688324722943e6fff1e6ab685` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:33.453833 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc87b58688324722943e6fff1e6ab685)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:33.455738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:33.584883 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc87b58688324722943e6fff1e6ab685', 'request_id': 'req-a8dd8dc2-41aa-438b-9348-a77dc8624b25'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:33.590964 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc87b58688324722943e6fff1e6ab685', 'request_id': 'req-a8dd8dc2-41aa-438b-9348-a77dc8624b25'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:33.597122 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc87b58688324722943e6fff1e6ab685', 'request_id': 'req-a8dd8dc2-41aa-438b-9348-a77dc8624b25'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:33.687203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] The token cache is being invalidate because project fc87b58688324722943e6fff1e6ab685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:17:33.687425 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:33.688042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:33.689632 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1419/2831] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:17:33 2022] DELETE /identity/v3/projects/fc87b58688324722943e6fff1e6ab685 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:40.960808 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda81fda-444f-4a48-a1b3-379aba7a96ec None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:40.961283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda81fda-444f-4a48-a1b3-379aba7a96ec None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:40.961466 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda81fda-444f-4a48-a1b3-379aba7a96ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:40.981053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-fda81fda-444f-4a48-a1b3-379aba7a96ec None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:40.982258 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fda81fda-444f-4a48-a1b3-379aba7a96ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:41.000073 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:41.037715 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1413/2832] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:17:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:41.216995 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:41.218430 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:41.286193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd8dc2-41aa-438b-9348-a77dc8624b25 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:41.291679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bd850-83c9-4aeb-bf3f-60bee0050c8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:41.292322 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bd850-83c9-4aeb-bf3f-60bee0050c8b service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:41.292604 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bd850-83c9-4aeb-bf3f-60bee0050c8b service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:41.293931 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bd850-83c9-4aeb-bf3f-60bee0050c8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:41.301972 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bd850-83c9-4aeb-bf3f-60bee0050c8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:41.303559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bd850-83c9-4aeb-bf3f-60bee0050c8b service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:41.312278 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1420/2833] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:17:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:17:42.116375 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d78a8-5144-4b60-b59b-c98686ee621d None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:42.116869 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d78a8-5144-4b60-b59b-c98686ee621d None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:42.116981 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d78a8-5144-4b60-b59b-c98686ee621d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:42.134973 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-788d78a8-5144-4b60-b59b-c98686ee621d None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:42.136155 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-788d78a8-5144-4b60-b59b-c98686ee621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:42.143777 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1414/2834] 10.222.0.45 () {58 vars in 967 bytes} [Tue Jul 5 16:17:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:42.155454 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bd850-83c9-4aeb-bf3f-60bee0050c8b service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:42.166228 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bd850-83c9-4aeb-bf3f-60bee0050c8b service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:42.171213 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10322e-c1ad-4039-81f9-1f7ba9e7bcca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:42.171844 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10322e-c1ad-4039-81f9-1f7ba9e7bcca service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:42.172130 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10322e-c1ad-4039-81f9-1f7ba9e7bcca service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:42.172408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10322e-c1ad-4039-81f9-1f7ba9e7bcca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:42.180387 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10322e-c1ad-4039-81f9-1f7ba9e7bcca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:42.181999 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10322e-c1ad-4039-81f9-1f7ba9e7bcca service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:42.187003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:42.190719 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1421/2835] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:17:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:17:42.851924 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9a8be-8d09-4b37-b301-5dbc360bc383 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:42.852213 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9a8be-8d09-4b37-b301-5dbc360bc383 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:42.852494 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9a8be-8d09-4b37-b301-5dbc360bc383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:42.870638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-79a9a8be-8d09-4b37-b301-5dbc360bc383 None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:42.871820 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-79a9a8be-8d09-4b37-b301-5dbc360bc383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:42.879010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1415/2836] 10.222.0.45 () {58 vars in 1009 bytes} [Tue Jul 5 16:17:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 16:17:42.890527 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10322e-c1ad-4039-81f9-1f7ba9e7bcca service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:42.901178 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10322e-c1ad-4039-81f9-1f7ba9e7bcca service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:42.906905 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462933f8-399d-45fc-a848-51d22458bc60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:42.907537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462933f8-399d-45fc-a848-51d22458bc60 service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:42.907820 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462933f8-399d-45fc-a848-51d22458bc60 service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:42.908096 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462933f8-399d-45fc-a848-51d22458bc60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:42.916183 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462933f8-399d-45fc-a848-51d22458bc60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:42.917814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462933f8-399d-45fc-a848-51d22458bc60 service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:42.926550 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1422/2837] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:17:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:17:43.501511 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa0c88-6e44-42fd-ab14-92837d994d10 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:43.501958 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa0c88-6e44-42fd-ab14-92837d994d10 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:43.502049 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa0c88-6e44-42fd-ab14-92837d994d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:43.519415 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-2eaa0c88-6e44-42fd-ab14-92837d994d10 None None] MFA Rules not processed for user `7245effe463b45dfa946ca2eda2e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:43.520641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2eaa0c88-6e44-42fd-ab14-92837d994d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:43.527986 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1416/2838] 10.222.0.45 () {58 vars in 967 bytes} [Tue Jul 5 16:17:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:43.539910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462933f8-399d-45fc-a848-51d22458bc60 service swift] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:43.551196 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462933f8-399d-45fc-a848-51d22458bc60 service swift] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:43.555957 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d281dc-e28d-450f-9e1c-7b96a6dfe09b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8da9cdd70af646928ee0edcadbcb524f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:43.556574 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d281dc-e28d-450f-9e1c-7b96a6dfe09b service swift] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:43.556891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d281dc-e28d-450f-9e1c-7b96a6dfe09b service swift] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:43.557165 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d281dc-e28d-450f-9e1c-7b96a6dfe09b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:43.565778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d281dc-e28d-450f-9e1c-7b96a6dfe09b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:43.567342 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d281dc-e28d-450f-9e1c-7b96a6dfe09b service swift] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:43.575311 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1423/2839] 10.222.0.45 () {60 vars in 1384 bytes} [Tue Jul 5 16:17:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 16:17:46.217310 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa0c88-6e44-42fd-ab14-92837d994d10 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:46.218742 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-2eaa0c88-6e44-42fd-ab14-92837d994d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:46.285707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa0c88-6e44-42fd-ab14-92837d994d10 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:46.290145 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:46.291008 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:46.292700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:46.292700 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] PATH_INFO: `/v3/users/0bb4d27ce2104d1086ef35180812b09e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:46.293307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb4d27ce2104d1086ef35180812b09e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:46.295718 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:46.477187 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb4d27ce2104d1086ef35180812b09e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:46.484485 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb4d27ce2104d1086ef35180812b09e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:46.485427 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb4d27ce2104d1086ef35180812b09e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:46.497595 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb4d27ce2104d1086ef35180812b09e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:46.498225 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb4d27ce2104d1086ef35180812b09e'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:46.561606 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1417/2840] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:17:46 2022] DELETE /identity/v3/users/0bb4d27ce2104d1086ef35180812b09e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:46.566126 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97c55b-2415-4473-864d-5c911e9b0c8a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:46.566611 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97c55b-2415-4473-864d-5c911e9b0c8a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:46.566705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97c55b-2415-4473-864d-5c911e9b0c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:46.584329 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-5b97c55b-2415-4473-864d-5c911e9b0c8a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:46.585577 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5b97c55b-2415-4473-864d-5c911e9b0c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:46.644291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1424/2841] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:46.654502 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:46.662738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:46.686952 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:46.712439 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278ec32-6b5d-447e-9d83-965288a6b2be admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:46.717441 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23273a72-7d02-4230-af7e-3f655269ab59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:46.718088 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23273a72-7d02-4230-af7e-3f655269ab59 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:46.718377 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23273a72-7d02-4230-af7e-3f655269ab59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:46.718651 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23273a72-7d02-4230-af7e-3f655269ab59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:46.727961 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23273a72-7d02-4230-af7e-3f655269ab59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:46.729354 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23273a72-7d02-4230-af7e-3f655269ab59 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:46.739518 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1418/2842] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:46.975143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97c55b-2415-4473-864d-5c911e9b0c8a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:46.975143 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5b97c55b-2415-4473-864d-5c911e9b0c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:46.990189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97c55b-2415-4473-864d-5c911e9b0c8a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:46.995072 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:46.995993 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:46.996276 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:46.996698 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] PATH_INFO: `/v3/projects/74d513d7cde3430f880b5dd5641d5356` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:46.998193 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d513d7cde3430f880b5dd5641d5356)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:47.000166 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:47.143422 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d513d7cde3430f880b5dd5641d5356', 'request_id': 'req-3711031d-8fd6-49ab-8ed7-e61a6af5d948'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:47.148973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d513d7cde3430f880b5dd5641d5356', 'request_id': 'req-3711031d-8fd6-49ab-8ed7-e61a6af5d948'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:47.155682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d513d7cde3430f880b5dd5641d5356', 'request_id': 'req-3711031d-8fd6-49ab-8ed7-e61a6af5d948'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:47.256442 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] The token cache is being invalidate because project 74d513d7cde3430f880b5dd5641d5356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:17:47.256676 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:47.257355 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3711031d-8fd6-49ab-8ed7-e61a6af5d948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:17:47.258888 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1425/2843] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:17:46 2022] DELETE /identity/v3/projects/74d513d7cde3430f880b5dd5641d5356 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:47.264228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3839d9-fc73-4768-8e73-18a95ab6487b None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:47.264705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3839d9-fc73-4768-8e73-18a95ab6487b None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:47.264847 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3839d9-fc73-4768-8e73-18a95ab6487b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:47.283110 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1e3839d9-fc73-4768-8e73-18a95ab6487b None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:47.287266 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1e3839d9-fc73-4768-8e73-18a95ab6487b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:47.343831 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1419/2844] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:47.352480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46eab1-e243-4ca1-b979-311a046302d6 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:47.352825 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46eab1-e243-4ca1-b979-311a046302d6 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:47.353110 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46eab1-e243-4ca1-b979-311a046302d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:47.371291 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-ff46eab1-e243-4ca1-b979-311a046302d6 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:47.373020 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-ff46eab1-e243-4ca1-b979-311a046302d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:47.380817 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1426/2845] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:47.384899 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3839d9-fc73-4768-8e73-18a95ab6487b None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:47.395136 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3839d9-fc73-4768-8e73-18a95ab6487b None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:47.399597 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab02ce-3c16-47b4-9d2b-ffc5736c27a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:47.400242 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab02ce-3c16-47b4-9d2b-ffc5736c27a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:47.400528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab02ce-3c16-47b4-9d2b-ffc5736c27a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:47.400839 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab02ce-3c16-47b4-9d2b-ffc5736c27a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:47.401564 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab02ce-3c16-47b4-9d2b-ffc5736c27a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:17:47.401897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab02ce-3c16-47b4-9d2b-ffc5736c27a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:47.403488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab02ce-3c16-47b4-9d2b-ffc5736c27a1 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:47.414938 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1420/2846] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 16:17:47.419012 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46eab1-e243-4ca1-b979-311a046302d6 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:47.428247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46eab1-e243-4ca1-b979-311a046302d6 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:47.432910 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358a39f-446b-45e3-9626-6ea4189f764c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:47.433554 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358a39f-446b-45e3-9626-6ea4189f764c admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:47.433841 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358a39f-446b-45e3-9626-6ea4189f764c admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:47.434148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358a39f-446b-45e3-9626-6ea4189f764c admin admin] PATH_INFO: `/v3/projects` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:47.435353 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358a39f-446b-45e3-9626-6ea4189f764c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:47.437270 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358a39f-446b-45e3-9626-6ea4189f764c admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:47.539891 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1427/2847] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:17:47 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:17:47.543680 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab02ce-3c16-47b4-9d2b-ffc5736c27a1 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:47.553541 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab02ce-3c16-47b4-9d2b-ffc5736c27a1 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:47.609586 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db0541-8cd6-4134-81b8-0e2f65fc9552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:47.610491 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db0541-8cd6-4134-81b8-0e2f65fc9552 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:47.610770 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db0541-8cd6-4134-81b8-0e2f65fc9552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:47.611042 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db0541-8cd6-4134-81b8-0e2f65fc9552 admin admin] PATH_INFO: `/v3/users` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:47.611723 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db0541-8cd6-4134-81b8-0e2f65fc9552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:47.613644 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db0541-8cd6-4134-81b8-0e2f65fc9552 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:47.687528 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1421/2848] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:17:47 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:17:47.691627 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358a39f-446b-45e3-9626-6ea4189f764c admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:47.702709 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358a39f-446b-45e3-9626-6ea4189f764c admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:47.707300 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585ce6e-d368-466d-9de4-e30d0e0929d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:47.707953 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ce6e-d368-466d-9de4-e30d0e0929d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:47.708240 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ce6e-d368-466d-9de4-e30d0e0929d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:47.708523 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ce6e-d368-466d-9de4-e30d0e0929d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:47.709289 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585ce6e-d368-466d-9de4-e30d0e0929d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:47.710899 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585ce6e-d368-466d-9de4-e30d0e0929d8 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:47.721200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1428/2849] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:17:47.725004 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db0541-8cd6-4134-81b8-0e2f65fc9552 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:47.735540 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db0541-8cd6-4134-81b8-0e2f65fc9552 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:47.740047 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d0d01-33aa-48f4-8b8a-64b9270767a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:47.740687 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d0d01-33aa-48f4-8b8a-64b9270767a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:47.741005 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d0d01-33aa-48f4-8b8a-64b9270767a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:47.741282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d0d01-33aa-48f4-8b8a-64b9270767a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:47.742030 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d0d01-33aa-48f4-8b8a-64b9270767a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:47.743592 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d0d01-33aa-48f4-8b8a-64b9270767a7 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:47.753790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1422/2850] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:17:47.757668 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585ce6e-d368-466d-9de4-e30d0e0929d8 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:47.767362 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585ce6e-d368-466d-9de4-e30d0e0929d8 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:47.771878 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15327013-379a-47b3-b012-e4d5c11d9a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:47.772739 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15327013-379a-47b3-b012-e4d5c11d9a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:47.773052 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15327013-379a-47b3-b012-e4d5c11d9a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:47.773328 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15327013-379a-47b3-b012-e4d5c11d9a4b admin admin] PATH_INFO: `/v3/projects/247d4348634044b9840ae86b7022d72c/users/e41741d00cf04063a21d163842ed5abe/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:47.787426 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15327013-379a-47b3-b012-e4d5c11d9a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=247d4348634044b9840ae86b7022d72c, user_id=e41741d00cf04063a21d163842ed5abe, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:47.790151 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15327013-379a-47b3-b012-e4d5c11d9a4b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:47.792302 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:47.902693 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1429/2851] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:17:47 2022] PUT /identity/v3/projects/247d4348634044b9840ae86b7022d72c/users/e41741d00cf04063a21d163842ed5abe/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:17:47.907642 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5538f34-a232-4b7e-a1a4-08126b14324a None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:47.907954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5538f34-a232-4b7e-a1a4-08126b14324a None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:47.908233 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5538f34-a232-4b7e-a1a4-08126b14324a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:47.925882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c5538f34-a232-4b7e-a1a4-08126b14324a None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:47.928050 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c5538f34-a232-4b7e-a1a4-08126b14324a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:47.956045 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:17:47.984679 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1423/2852] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:47.994448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15327013-379a-47b3-b012-e4d5c11d9a4b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:47.996119 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-15327013-379a-47b3-b012-e4d5c11d9a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:48.053244 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15327013-379a-47b3-b012-e4d5c11d9a4b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:48.056705 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df86e24-e83f-4d80-addf-92f022af3699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:48.057395 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df86e24-e83f-4d80-addf-92f022af3699 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:48.057652 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df86e24-e83f-4d80-addf-92f022af3699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:48.057928 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df86e24-e83f-4d80-addf-92f022af3699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:48.067440 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df86e24-e83f-4d80-addf-92f022af3699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:48.068917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df86e24-e83f-4d80-addf-92f022af3699 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:48.078326 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1430/2853] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:49.327471 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97231d-cf5a-451d-9bca-6440fb4eea71 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:49.327977 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97231d-cf5a-451d-9bca-6440fb4eea71 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:49.328074 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97231d-cf5a-451d-9bca-6440fb4eea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:49.354837 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-bd97231d-cf5a-451d-9bca-6440fb4eea71 None None] MFA Rules not processed for user `e41741d00cf04063a21d163842ed5abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:49.356557 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-bd97231d-cf5a-451d-9bca-6440fb4eea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:49.420946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1424/2854] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:49.431769 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df86e24-e83f-4d80-addf-92f022af3699 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:17:49.433188 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5df86e24-e83f-4d80-addf-92f022af3699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:49.493229 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df86e24-e83f-4d80-addf-92f022af3699 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:17:49.497374 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230ad7e-6b66-46a9-93b9-860153ad6700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:17:49.498011 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230ad7e-6b66-46a9-93b9-860153ad6700 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:49.498292 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230ad7e-6b66-46a9-93b9-860153ad6700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:49.498567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230ad7e-6b66-46a9-93b9-860153ad6700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:49.506682 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230ad7e-6b66-46a9-93b9-860153ad6700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:17:49.508105 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230ad7e-6b66-46a9-93b9-860153ad6700 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:17:49.516711 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1431/2855] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:17:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:17:51.363453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb86725-7b36-413b-a534-ca53a59f1dce None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:17:51.363971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb86725-7b36-413b-a534-ca53a59f1dce None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:17:51.364097 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb86725-7b36-413b-a534-ca53a59f1dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:17:51.381303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-4fb86725-7b36-413b-a534-ca53a59f1dce None None] MFA Rules not processed for user `e41741d00cf04063a21d163842ed5abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:17:51.382510 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-4fb86725-7b36-413b-a534-ca53a59f1dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:17:51.389343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1425/2856] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:17:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:17:59.512992 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=1,keystone:SELECT=54 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:18:01.376228 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=46 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:18:44.204148 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230ad7e-6b66-46a9-93b9-860153ad6700 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:18:44.212806 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230ad7e-6b66-46a9-93b9-860153ad6700 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:18:44.216386 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32939d80-7dee-44c0-9005-7e80b5852301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:18:44.217058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939d80-7dee-44c0-9005-7e80b5852301 service cinder] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:18:44.217340 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939d80-7dee-44c0-9005-7e80b5852301 service cinder] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:18:44.217614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32939d80-7dee-44c0-9005-7e80b5852301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:18:44.219371 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-32939d80-7dee-44c0-9005-7e80b5852301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:18:44.413476 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32939d80-7dee-44c0-9005-7e80b5852301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:18:44.415629 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32939d80-7dee-44c0-9005-7e80b5852301 service cinder] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:18:44.426600 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1432/2857] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:18:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:18:54.421559 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=34 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:18:58.147011 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c876053-3663-42ea-a248-26e8001a0397 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:18:58.147573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c876053-3663-42ea-a248-26e8001a0397 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:18:58.147573 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c876053-3663-42ea-a248-26e8001a0397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:18:58.165587 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-1c876053-3663-42ea-a248-26e8001a0397 None None] MFA Rules not processed for user `e41741d00cf04063a21d163842ed5abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:18:58.166779 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1c876053-3663-42ea-a248-26e8001a0397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:18:58.173620 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1426/2858] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:18:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:19:08.160479 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:19:36.855911 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32939d80-7dee-44c0-9005-7e80b5852301 service cinder] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:19:36.857344 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-32939d80-7dee-44c0-9005-7e80b5852301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:19:36.871952 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32939d80-7dee-44c0-9005-7e80b5852301 service cinder] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:19:36.877401 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395ddce-5fcc-4e9e-831b-d1a95be86068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:19:36.878035 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395ddce-5fcc-4e9e-831b-d1a95be86068 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:19:36.878318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395ddce-5fcc-4e9e-831b-d1a95be86068 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:19:36.878593 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395ddce-5fcc-4e9e-831b-d1a95be86068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:19:36.880305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-9395ddce-5fcc-4e9e-831b-d1a95be86068 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:19:36.895272 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395ddce-5fcc-4e9e-831b-d1a95be86068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:19:36.897731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395ddce-5fcc-4e9e-831b-d1a95be86068 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:19:36.913203 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1433/2859] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:19:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:19:46.906528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:20:01.736968 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c876053-3663-42ea-a248-26e8001a0397 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:01.747232 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c876053-3663-42ea-a248-26e8001a0397 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:01.751107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97c86f-1228-4edd-a090-e50b96228bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:01.751738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97c86f-1228-4edd-a090-e50b96228bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:01.752017 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97c86f-1228-4edd-a090-e50b96228bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:01.752290 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97c86f-1228-4edd-a090-e50b96228bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:01.754116 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-9f97c86f-1228-4edd-a090-e50b96228bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:01.763517 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97c86f-1228-4edd-a090-e50b96228bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:01.764954 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97c86f-1228-4edd-a090-e50b96228bc8 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:01.773738 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1427/2860] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:20:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:20:02.827893 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395ddce-5fcc-4e9e-831b-d1a95be86068 service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:02.837926 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395ddce-5fcc-4e9e-831b-d1a95be86068 service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:02.842388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:02.843247 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:02.843516 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:02.843787 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] PATH_INFO: `/v3/users/e41741d00cf04063a21d163842ed5abe` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:02.844989 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41741d00cf04063a21d163842ed5abe)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:02.850655 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:03.022566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41741d00cf04063a21d163842ed5abe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:03.029752 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41741d00cf04063a21d163842ed5abe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:03.030401 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41741d00cf04063a21d163842ed5abe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:03.031040 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41741d00cf04063a21d163842ed5abe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:03.043201 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-b3b01bae-5710-47c1-a35f-b66b5f4a2513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41741d00cf04063a21d163842ed5abe'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:03.094822 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1434/2861] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:20:02 2022] DELETE /identity/v3/users/e41741d00cf04063a21d163842ed5abe => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:20:03.308272 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97c86f-1228-4edd-a090-e50b96228bc8 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:03.323307 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97c86f-1228-4edd-a090-e50b96228bc8 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:03.386717 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:03.387582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:03.387864 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:03.388138 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] PATH_INFO: `/v3/projects/247d4348634044b9840ae86b7022d72c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:03.389406 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=247d4348634044b9840ae86b7022d72c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:03.391300 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:03.488691 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '247d4348634044b9840ae86b7022d72c', 'request_id': 'req-1a02ffb3-a027-4251-ad54-22729f1f3728'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:03.494821 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '247d4348634044b9840ae86b7022d72c', 'request_id': 'req-1a02ffb3-a027-4251-ad54-22729f1f3728'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:03.546214 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '247d4348634044b9840ae86b7022d72c', 'request_id': 'req-1a02ffb3-a027-4251-ad54-22729f1f3728'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:03.557163 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] The token cache is being invalidate because project 247d4348634044b9840ae86b7022d72c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:20:03.557453 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:03.558077 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-1a02ffb3-a027-4251-ad54-22729f1f3728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:03.559631 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1428/2862] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:20:03 2022] DELETE /identity/v3/projects/247d4348634044b9840ae86b7022d72c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:20:03.564546 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde92c4-5206-4c65-9866-b9141519f9ca None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:03.564870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde92c4-5206-4c65-9866-b9141519f9ca None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:03.565150 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde92c4-5206-4c65-9866-b9141519f9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:03.583444 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-cbde92c4-5206-4c65-9866-b9141519f9ca None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:20:03.584637 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-cbde92c4-5206-4c65-9866-b9141519f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:03.641990 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1435/2863] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:20:03.650774 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b046bc-e9f2-4987-bf6d-26749d8684ce None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:03.651135 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b046bc-e9f2-4987-bf6d-26749d8684ce None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:03.651421 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b046bc-e9f2-4987-bf6d-26749d8684ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:03.669525 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-71b046bc-e9f2-4987-bf6d-26749d8684ce None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:20:03.670783 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-71b046bc-e9f2-4987-bf6d-26749d8684ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:03.678638 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1429/2864] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:20:03.682773 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde92c4-5206-4c65-9866-b9141519f9ca None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:03.692273 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde92c4-5206-4c65-9866-b9141519f9ca None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:03.697377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4381a2-2452-4f7c-a21c-34e94350c0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:03.698022 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4381a2-2452-4f7c-a21c-34e94350c0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:03.698308 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4381a2-2452-4f7c-a21c-34e94350c0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:03.698587 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4381a2-2452-4f7c-a21c-34e94350c0bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:03.699319 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4381a2-2452-4f7c-a21c-34e94350c0bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 16:20:03.699622 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4381a2-2452-4f7c-a21c-34e94350c0bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:03.701258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4381a2-2452-4f7c-a21c-34e94350c0bd admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:03.713894 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1436/2865] 10.222.0.45 () {58 vars in 1152 bytes} [Tue Jul 5 16:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 16:20:03.717988 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b046bc-e9f2-4987-bf6d-26749d8684ce None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:03.727285 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b046bc-e9f2-4987-bf6d-26749d8684ce None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:03.731971 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d281254-f8d6-4d19-acab-30518b82c579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:03.732608 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d281254-f8d6-4d19-acab-30518b82c579 admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:03.732929 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d281254-f8d6-4d19-acab-30518b82c579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:03.733207 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d281254-f8d6-4d19-acab-30518b82c579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:03.733920 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d281254-f8d6-4d19-acab-30518b82c579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:03.736151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d281254-f8d6-4d19-acab-30518b82c579 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:03.804456 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1430/2866] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:20:03 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:20:03.808895 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4381a2-2452-4f7c-a21c-34e94350c0bd admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:03.819112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4381a2-2452-4f7c-a21c-34e94350c0bd admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:03.883874 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6503cf-3108-4e84-8549-581fa7a43319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:03.885528 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6503cf-3108-4e84-8549-581fa7a43319 admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:03.885960 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6503cf-3108-4e84-8549-581fa7a43319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:03.886380 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6503cf-3108-4e84-8549-581fa7a43319 admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:03.887448 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6503cf-3108-4e84-8549-581fa7a43319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:03.890567 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6503cf-3108-4e84-8549-581fa7a43319 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:03.897501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:20:03.963372 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1437/2867] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:20:03 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:20:03.967381 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d281254-f8d6-4d19-acab-30518b82c579 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:03.977692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d281254-f8d6-4d19-acab-30518b82c579 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:03.982291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5eb72-e540-4bc4-aeeb-3bf3a9a89715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:03.982934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5eb72-e540-4bc4-aeeb-3bf3a9a89715 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:03.983211 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5eb72-e540-4bc4-aeeb-3bf3a9a89715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:03.983484 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5eb72-e540-4bc4-aeeb-3bf3a9a89715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:03.984195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5eb72-e540-4bc4-aeeb-3bf3a9a89715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:03.986069 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5eb72-e540-4bc4-aeeb-3bf3a9a89715 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:03.997053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1431/2868] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:20:04.000716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6503cf-3108-4e84-8549-581fa7a43319 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:04.011157 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6503cf-3108-4e84-8549-581fa7a43319 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:04.015811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f943dea-0eb0-488b-8a26-6cff862055e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:04.016458 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943dea-0eb0-488b-8a26-6cff862055e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:04.016743 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943dea-0eb0-488b-8a26-6cff862055e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:04.017058 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943dea-0eb0-488b-8a26-6cff862055e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:04.017782 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f943dea-0eb0-488b-8a26-6cff862055e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:04.019367 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f943dea-0eb0-488b-8a26-6cff862055e4 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:04.029267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1438/2869] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 16:20:04.033309 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5eb72-e540-4bc4-aeeb-3bf3a9a89715 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:04.043053 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5eb72-e540-4bc4-aeeb-3bf3a9a89715 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:04.047543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cec1b-08f0-4309-9703-f15984dbb73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:04.048403 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cec1b-08f0-4309-9703-f15984dbb73b admin admin] REQUEST_METHOD: `PUT` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:04.048692 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cec1b-08f0-4309-9703-f15984dbb73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:04.049013 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cec1b-08f0-4309-9703-f15984dbb73b admin admin] PATH_INFO: `/v3/projects/85bf8c1d40a04aaf9e46cd7f1e58048a/users/b1cf6897a2a74b7c84b26bae868845f4/roles/61e62f29eafd45c8b9fb99205a60bc4e` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:04.063740 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cec1b-08f0-4309-9703-f15984dbb73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=85bf8c1d40a04aaf9e46cd7f1e58048a, user_id=b1cf6897a2a74b7c84b26bae868845f4, role_id=61e62f29eafd45c8b9fb99205a60bc4e)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:04.066329 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cec1b-08f0-4309-9703-f15984dbb73b admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:04.169589 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1432/2870] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:20:04 2022] PUT /identity/v3/projects/85bf8c1d40a04aaf9e46cd7f1e58048a/users/b1cf6897a2a74b7c84b26bae868845f4/roles/61e62f29eafd45c8b9fb99205a60bc4e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:20:04.180504 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0738655-f7e4-4c0f-9f81-f0d463eb9ff9 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:04.180815 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0738655-f7e4-4c0f-9f81-f0d463eb9ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:04.181778 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0738655-f7e4-4c0f-9f81-f0d463eb9ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:04.199544 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-c0738655-f7e4-4c0f-9f81-f0d463eb9ff9 None None] MFA Rules not processed for user `45ceaef67f814339b45f2d297054bedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:20:04.200727 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c0738655-f7e4-4c0f-9f81-f0d463eb9ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:04.259764 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1439/2871] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:20:04.269550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cec1b-08f0-4309-9703-f15984dbb73b admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:04.271255 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-fb9cec1b-08f0-4309-9703-f15984dbb73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:04.285469 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:20:04.330521 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cec1b-08f0-4309-9703-f15984dbb73b admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:04.334107 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264dfcb-8a83-4042-985c-56e45ce1b9f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:04.334741 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264dfcb-8a83-4042-985c-56e45ce1b9f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:04.335021 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264dfcb-8a83-4042-985c-56e45ce1b9f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:04.335295 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264dfcb-8a83-4042-985c-56e45ce1b9f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:04.344757 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264dfcb-8a83-4042-985c-56e45ce1b9f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:04.346219 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264dfcb-8a83-4042-985c-56e45ce1b9f9 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:04.355900 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1433/2872] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:20:09.514260 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c983a-d563-4ef0-a2ee-2641a70543df None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:09.515212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c983a-d563-4ef0-a2ee-2641a70543df None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:09.515212 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c983a-d563-4ef0-a2ee-2641a70543df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:09.544802 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-0f4c983a-d563-4ef0-a2ee-2641a70543df None None] MFA Rules not processed for user `b1cf6897a2a74b7c84b26bae868845f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:20:09.546137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-0f4c983a-d563-4ef0-a2ee-2641a70543df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:09.617112 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1440/2873] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:20:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:20:09.622051 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264dfcb-8a83-4042-985c-56e45ce1b9f9 service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:09.632495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264dfcb-8a83-4042-985c-56e45ce1b9f9 service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:09.637407 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b107e25-7868-44f5-b011-6570cf63bdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:09.641728 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b107e25-7868-44f5-b011-6570cf63bdeb admin admin] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:09.641728 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b107e25-7868-44f5-b011-6570cf63bdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:09.641728 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b107e25-7868-44f5-b011-6570cf63bdeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:09.641728 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b107e25-7868-44f5-b011-6570cf63bdeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:09.641728 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b107e25-7868-44f5-b011-6570cf63bdeb admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:09.768344 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1434/2874] 10.222.0.45 () {60 vars in 1152 bytes} [Tue Jul 5 16:20:09 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:20:09.769695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c983a-d563-4ef0-a2ee-2641a70543df None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:09.780949 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c983a-d563-4ef0-a2ee-2641a70543df None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:09.838916 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c8907-0a57-404b-a172-ba53f195ebdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:09.839605 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c8907-0a57-404b-a172-ba53f195ebdc admin admin] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:09.839884 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c8907-0a57-404b-a172-ba53f195ebdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:09.840156 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c8907-0a57-404b-a172-ba53f195ebdc admin admin] PATH_INFO: `/v3/users` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:09.840880 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c8907-0a57-404b-a172-ba53f195ebdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:09.842791 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c8907-0a57-404b-a172-ba53f195ebdc admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:09.924210 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:20:09.930689 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1441/2875] 10.222.0.45 () {60 vars in 1143 bytes} [Tue Jul 5 16:20:09 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 16:20:09.934617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b107e25-7868-44f5-b011-6570cf63bdeb admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:09.945732 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b107e25-7868-44f5-b011-6570cf63bdeb admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:09.950339 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b06e86-b2e2-45f3-b111-72d6992c8480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:09.950998 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b06e86-b2e2-45f3-b111-72d6992c8480 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:09.951283 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b06e86-b2e2-45f3-b111-72d6992c8480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:09.951562 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b06e86-b2e2-45f3-b111-72d6992c8480 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:09.952291 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b06e86-b2e2-45f3-b111-72d6992c8480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:09.953904 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b06e86-b2e2-45f3-b111-72d6992c8480 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:09.964849 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1435/2876] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:20:09.968821 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c8907-0a57-404b-a172-ba53f195ebdc admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:09.979581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c8907-0a57-404b-a172-ba53f195ebdc admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:09.984347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c51bcc-63d5-4f53-937a-064939aeb74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:09.985592 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c51bcc-63d5-4f53-937a-064939aeb74a admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:09.986089 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c51bcc-63d5-4f53-937a-064939aeb74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:09.986537 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c51bcc-63d5-4f53-937a-064939aeb74a admin admin] PATH_INFO: `/v3/projects/62a2450c58494ce0a452354e69162dd2/users/24f0920000684a17992bc96f90278317/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:10.002200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c51bcc-63d5-4f53-937a-064939aeb74a admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a2450c58494ce0a452354e69162dd2, user_id=24f0920000684a17992bc96f90278317, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:10.004881 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c51bcc-63d5-4f53-937a-064939aeb74a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:10.095811 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1442/2877] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:20:09 2022] PUT /identity/v3/projects/62a2450c58494ce0a452354e69162dd2/users/24f0920000684a17992bc96f90278317/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:20:10.099763 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b06e86-b2e2-45f3-b111-72d6992c8480 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:10.110776 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b06e86-b2e2-45f3-b111-72d6992c8480 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:10.166151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fbe42-cd0e-46b9-8863-338a21751853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:10.166814 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fbe42-cd0e-46b9-8863-338a21751853 admin admin] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:10.167100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fbe42-cd0e-46b9-8863-338a21751853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:10.167378 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fbe42-cd0e-46b9-8863-338a21751853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:10.168099 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fbe42-cd0e-46b9-8863-338a21751853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:10.169711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fbe42-cd0e-46b9-8863-338a21751853 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:10.179263 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1436/2878] 10.222.0.45 () {58 vars in 1121 bytes} [Tue Jul 5 16:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 16:20:10.183408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c51bcc-63d5-4f53-937a-064939aeb74a admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:10.197810 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c51bcc-63d5-4f53-937a-064939aeb74a admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:10.203512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e88d1-190f-4347-bbff-925390790f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:10.204489 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e88d1-190f-4347-bbff-925390790f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:10.204674 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e88d1-190f-4347-bbff-925390790f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:10.205005 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e88d1-190f-4347-bbff-925390790f20 admin admin] PATH_INFO: `/v3/projects/62a2450c58494ce0a452354e69162dd2/users/24f0920000684a17992bc96f90278317/roles/c124ec42c56b4f43bb474fcf5e9b89b3` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:10.208787 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e88d1-190f-4347-bbff-925390790f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a2450c58494ce0a452354e69162dd2, user_id=24f0920000684a17992bc96f90278317, role_id=c124ec42c56b4f43bb474fcf5e9b89b3)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:10.211780 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e88d1-190f-4347-bbff-925390790f20 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:10.226954 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1443/2879] 10.222.0.45 () {60 vars in 1482 bytes} [Tue Jul 5 16:20:10 2022] PUT /identity/v3/projects/62a2450c58494ce0a452354e69162dd2/users/24f0920000684a17992bc96f90278317/roles/c124ec42c56b4f43bb474fcf5e9b89b3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:20:15.391003 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53df87a-50b5-49d9-a05d-57e355aaea09 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:15.391508 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53df87a-50b5-49d9-a05d-57e355aaea09 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:15.391611 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53df87a-50b5-49d9-a05d-57e355aaea09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:15.420452 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-c53df87a-50b5-49d9-a05d-57e355aaea09 None None] MFA Rules not processed for user `24f0920000684a17992bc96f90278317`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:20:15.421792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-c53df87a-50b5-49d9-a05d-57e355aaea09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:15.491792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1437/2880] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:20:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:20:15.505545 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e88d1-190f-4347-bbff-925390790f20 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:15.507305 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-112e88d1-190f-4347-bbff-925390790f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:15.586479 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e88d1-190f-4347-bbff-925390790f20 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:15.592570 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065df088-fe20-449e-bbec-bd60a6febf01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:15.593257 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065df088-fe20-449e-bbec-bd60a6febf01 service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:15.593539 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065df088-fe20-449e-bbec-bd60a6febf01 service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:15.593814 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065df088-fe20-449e-bbec-bd60a6febf01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:15.652031 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065df088-fe20-449e-bbec-bd60a6febf01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:15.653756 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065df088-fe20-449e-bbec-bd60a6febf01 service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:15.664043 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1444/2881] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:20:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:20:25.488745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:20:25.659280 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:20:33.137303 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53df87a-50b5-49d9-a05d-57e355aaea09 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:33.151282 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53df87a-50b5-49d9-a05d-57e355aaea09 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:33.156855 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979e497-9185-4ee2-b31d-c40ca2db7a66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:33.157505 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979e497-9185-4ee2-b31d-c40ca2db7a66 service nova] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:33.157790 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979e497-9185-4ee2-b31d-c40ca2db7a66 service nova] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:33.158068 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979e497-9185-4ee2-b31d-c40ca2db7a66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:33.160188 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e979e497-9185-4ee2-b31d-c40ca2db7a66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:33.172928 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979e497-9185-4ee2-b31d-c40ca2db7a66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:33.174506 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979e497-9185-4ee2-b31d-c40ca2db7a66 service nova] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:33.185673 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1438/2882] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:20:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:20:33.709716 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8502b9-4d08-440f-8f5b-5e5673d2829a None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:33.710255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8502b9-4d08-440f-8f5b-5e5673d2829a None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:33.710255 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8502b9-4d08-440f-8f5b-5e5673d2829a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:33.727416 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-df8502b9-4d08-440f-8f5b-5e5673d2829a None None] MFA Rules not processed for user `b1cf6897a2a74b7c84b26bae868845f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:20:33.729204 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df8502b9-4d08-440f-8f5b-5e5673d2829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:33.736364 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1445/2883] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:20:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:20:34.040646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979e497-9185-4ee2-b31d-c40ca2db7a66 service nova] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:34.042120 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-e979e497-9185-4ee2-b31d-c40ca2db7a66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:34.098041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979e497-9185-4ee2-b31d-c40ca2db7a66 service nova] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:34.101703 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1628527-919e-4139-8c95-914b27a246af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:34.102343 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628527-919e-4139-8c95-914b27a246af service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:34.102629 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628527-919e-4139-8c95-914b27a246af service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:34.102905 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628527-919e-4139-8c95-914b27a246af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:34.104661 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a1628527-919e-4139-8c95-914b27a246af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:34.114301 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1628527-919e-4139-8c95-914b27a246af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:34.115750 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1628527-919e-4139-8c95-914b27a246af service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:34.124195 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1439/2884] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:20:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:20:34.871500 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8502b9-4d08-440f-8f5b-5e5673d2829a None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:34.873007 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-df8502b9-4d08-440f-8f5b-5e5673d2829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:34.946695 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8502b9-4d08-440f-8f5b-5e5673d2829a None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:34.951939 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137258a-e64b-4a72-bbba-133592bd1b82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:34.952600 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137258a-e64b-4a72-bbba-133592bd1b82 service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:34.952917 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137258a-e64b-4a72-bbba-133592bd1b82 service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:34.953191 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137258a-e64b-4a72-bbba-133592bd1b82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:34.954977 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b137258a-e64b-4a72-bbba-133592bd1b82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:34.968038 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137258a-e64b-4a72-bbba-133592bd1b82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:34.969679 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137258a-e64b-4a72-bbba-133592bd1b82 service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:34.981312 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1446/2885] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:20:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:20:41.656838 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1628527-919e-4139-8c95-914b27a246af service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:41.658337 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a1628527-919e-4139-8c95-914b27a246af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:41.675524 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1628527-919e-4139-8c95-914b27a246af service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:41.678536 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bab5d-dc85-4593-89a1-965aa3530910 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:41.679158 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bab5d-dc85-4593-89a1-965aa3530910 service placement] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:41.679437 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bab5d-dc85-4593-89a1-965aa3530910 service placement] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:41.679713 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bab5d-dc85-4593-89a1-965aa3530910 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:41.682067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-a49bab5d-dc85-4593-89a1-965aa3530910 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:41.693786 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bab5d-dc85-4593-89a1-965aa3530910 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:41.695349 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bab5d-dc85-4593-89a1-965aa3530910 service placement] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:41.706413 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1440/2886] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:20:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:20:44.975770 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=35 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:20:45.479641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137258a-e64b-4a72-bbba-133592bd1b82 service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:45.479641 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-b137258a-e64b-4a72-bbba-133592bd1b82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:45.542116 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137258a-e64b-4a72-bbba-133592bd1b82 service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:45.546512 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:45.547359 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:45.547634 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:45.547902 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] PATH_INFO: `/v3/users/fe5390b9df9b445380119b995f4c457d` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:45.549107 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5390b9df9b445380119b995f4c457d)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:45.551016 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:45.787700 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5390b9df9b445380119b995f4c457d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:45.794446 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5390b9df9b445380119b995f4c457d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:45.795118 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5390b9df9b445380119b995f4c457d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:45.795706 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5390b9df9b445380119b995f4c457d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:45.807283 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5390b9df9b445380119b995f4c457d'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:45.888339 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1447/2887] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:20:45 2022] DELETE /identity/v3/users/fe5390b9df9b445380119b995f4c457d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:20:46.155775 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bab5d-dc85-4593-89a1-965aa3530910 service placement] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:46.166659 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bab5d-dc85-4593-89a1-965aa3530910 service placement] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:46.231133 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:46.231995 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:46.232281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:46.232688 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] PATH_INFO: `/v3/projects/de07093c553b4f0391c85e5ea04cf7f1` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:46.234041 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de07093c553b4f0391c85e5ea04cf7f1)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:46.236252 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:46.330399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de07093c553b4f0391c85e5ea04cf7f1', 'request_id': 'req-7f932d71-c053-4da8-834c-041069695313'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:46.335144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de07093c553b4f0391c85e5ea04cf7f1', 'request_id': 'req-7f932d71-c053-4da8-834c-041069695313'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:46.390583 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de07093c553b4f0391c85e5ea04cf7f1', 'request_id': 'req-7f932d71-c053-4da8-834c-041069695313'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:46.401260 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] The token cache is being invalidate because project de07093c553b4f0391c85e5ea04cf7f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:20:46.401563 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:46.402278 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:46.403792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1441/2888] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:20:46 2022] DELETE /identity/v3/projects/de07093c553b4f0391c85e5ea04cf7f1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:20:46.407882 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:46.409324 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:46.422421 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497f10f-a6da-4af8-81d2-090737f30410 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:46.477921 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:46.478781 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:46.479061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:46.479332 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] PATH_INFO: `/v3/users/ab59bfdbbf9b4e48a4cee3b9b85006e2` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:46.480543 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab59bfdbbf9b4e48a4cee3b9b85006e2)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:46.482480 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:46.505152 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:20:46.701200 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab59bfdbbf9b4e48a4cee3b9b85006e2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:46.707857 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab59bfdbbf9b4e48a4cee3b9b85006e2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:46.708509 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab59bfdbbf9b4e48a4cee3b9b85006e2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:46.709135 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab59bfdbbf9b4e48a4cee3b9b85006e2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:46.720511 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab59bfdbbf9b4e48a4cee3b9b85006e2'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:46.813239 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1448/2889] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:20:46 2022] DELETE /identity/v3/users/ab59bfdbbf9b4e48a4cee3b9b85006e2 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:20:47.032868 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:47.034280 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:47.049488 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f932d71-c053-4da8-834c-041069695313 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:47.052424 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:20:47.106675 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:47.107543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:47.107825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:47.108192 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] PATH_INFO: `/v3/projects/029709f656f44d5d999793a45eef802c` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:47.109401 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=029709f656f44d5d999793a45eef802c)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:47.111550 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:47.238190 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '029709f656f44d5d999793a45eef802c', 'request_id': 'req-511e64a0-eecc-4ecf-93e3-14df6dbcd280'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:47.244593 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '029709f656f44d5d999793a45eef802c', 'request_id': 'req-511e64a0-eecc-4ecf-93e3-14df6dbcd280'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:47.317151 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '029709f656f44d5d999793a45eef802c', 'request_id': 'req-511e64a0-eecc-4ecf-93e3-14df6dbcd280'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:47.328897 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] The token cache is being invalidate because project 029709f656f44d5d999793a45eef802c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:20:47.329186 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:47.329860 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:20:47.331605 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1442/2890] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:20:47 2022] DELETE /identity/v3/projects/029709f656f44d5d999793a45eef802c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:20:50.065469 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:50.067137 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:50.146647 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de43874-b7d9-4b1b-9acb-17dd0090516b admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:50.211320 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d150979-d165-4a7d-ad4d-48eb2170334e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:50.212471 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150979-d165-4a7d-ad4d-48eb2170334e service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:50.212751 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150979-d165-4a7d-ad4d-48eb2170334e service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:50.213049 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150979-d165-4a7d-ad4d-48eb2170334e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:50.214831 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1d150979-d165-4a7d-ad4d-48eb2170334e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:50.226845 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d150979-d165-4a7d-ad4d-48eb2170334e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:50.229061 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d150979-d165-4a7d-ad4d-48eb2170334e service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:50.239738 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1449/2891] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:20:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:20:53.416601 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:20:53.418177 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:53.477245 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e64a0-eecc-4ecf-93e3-14df6dbcd280 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:20:53.480870 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562da105-2528-4525-815a-0ce19f47a132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:20:53.481509 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562da105-2528-4525-815a-0ce19f47a132 service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:20:53.481792 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562da105-2528-4525-815a-0ce19f47a132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:20:53.482067 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562da105-2528-4525-815a-0ce19f47a132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:20:53.483819 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-562da105-2528-4525-815a-0ce19f47a132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:20:53.533130 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:20:53.546537 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562da105-2528-4525-815a-0ce19f47a132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:20:53.547951 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562da105-2528-4525-815a-0ce19f47a132 service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:20:53.557240 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1443/2892] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:20:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:21:00.235019 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:DELETE=7,keystone:SELECT=82,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:21:03.716400 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=6 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:22:07.968943 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d150979-d165-4a7d-ad4d-48eb2170334e service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:22:07.970365 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-1d150979-d165-4a7d-ad4d-48eb2170334e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:22:08.112414 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d150979-d165-4a7d-ad4d-48eb2170334e service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:22:08.117639 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a19b9-552f-4142-b608-4038cb8fdd0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:22:08.118994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a19b9-552f-4142-b608-4038cb8fdd0b service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:22:08.118994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a19b9-552f-4142-b608-4038cb8fdd0b service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:22:08.118994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a19b9-552f-4142-b608-4038cb8fdd0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:22:08.120630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-5c2a19b9-552f-4142-b608-4038cb8fdd0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:22:08.132470 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a19b9-552f-4142-b608-4038cb8fdd0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:22:08.134080 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a19b9-552f-4142-b608-4038cb8fdd0b service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:22:08.145398 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1450/2893] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:22:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:22:18.140731 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=63 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:22:54.698946 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca2b1f-82ef-4bb8-9b20-ff25c34cc7a8 None None] REQUEST_METHOD: `POST` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:22:54.699423 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca2b1f-82ef-4bb8-9b20-ff25c34cc7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:22:54.699529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca2b1f-82ef-4bb8-9b20-ff25c34cc7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:22:54.717144 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.auth.core [None req-7dca2b1f-82ef-4bb8-9b20-ff25c34cc7a8 None None] MFA Rules not processed for user `b1cf6897a2a74b7c84b26bae868845f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:22:54.718326 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7dca2b1f-82ef-4bb8-9b20-ff25c34cc7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:22:54.772153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1444/2894] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:22:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:23:04.765711 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=25 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:24:49.828912 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28186844-d3f2-4d32-88f5-a4774721f271 None None] REQUEST_METHOD: `POST` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:24:49.829418 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28186844-d3f2-4d32-88f5-a4774721f271 None None] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:24:49.829526 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28186844-d3f2-4d32-88f5-a4774721f271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:24:49.846692 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.auth.core [None req-28186844-d3f2-4d32-88f5-a4774721f271 None None] MFA Rules not processed for user `b1cf6897a2a74b7c84b26bae868845f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 16:24:49.847863 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-28186844-d3f2-4d32-88f5-a4774721f271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:24:49.854673 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1451/2895] 10.222.0.45 () {58 vars in 957 bytes} [Tue Jul 5 16:24:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 16:24:49.892582 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca2b1f-82ef-4bb8-9b20-ff25c34cc7a8 None None] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:24:49.893999 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-7dca2b1f-82ef-4bb8-9b20-ff25c34cc7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:24:49.969867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca2b1f-82ef-4bb8-9b20-ff25c34cc7a8 None None] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:24:49.973455 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c095408-0aa3-402b-93cf-e78fbed7c5de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '805a9dd2d0934359b7ca5fe629b96d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:24:49.974090 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c095408-0aa3-402b-93cf-e78fbed7c5de service cinder] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:24:49.974371 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c095408-0aa3-402b-93cf-e78fbed7c5de service cinder] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:24:49.974646 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c095408-0aa3-402b-93cf-e78fbed7c5de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:24:49.976495 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-1c095408-0aa3-402b-93cf-e78fbed7c5de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:24:50.008181 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c095408-0aa3-402b-93cf-e78fbed7c5de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:24:50.009551 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c095408-0aa3-402b-93cf-e78fbed7c5de service cinder] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:24:50.029955 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1445/2896] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:24:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:24:54.533566 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28186844-d3f2-4d32-88f5-a4774721f271 None None] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:24:54.535521 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-28186844-d3f2-4d32-88f5-a4774721f271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:24:54.562501 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28186844-d3f2-4d32-88f5-a4774721f271 None None] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:24:54.568303 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce002e-9bf1-47ff-b9fd-1bb6ef83a87b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef95ab7b924d456195a8761e894d911b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:24:54.568994 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce002e-9bf1-47ff-b9fd-1bb6ef83a87b service nova] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:24:54.569304 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce002e-9bf1-47ff-b9fd-1bb6ef83a87b service nova] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:24:54.569580 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce002e-9bf1-47ff-b9fd-1bb6ef83a87b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:24:54.571360 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-c2ce002e-9bf1-47ff-b9fd-1bb6ef83a87b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:24:54.584377 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce002e-9bf1-47ff-b9fd-1bb6ef83a87b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:24:54.589136 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce002e-9bf1-47ff-b9fd-1bb6ef83a87b service nova] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:24:54.597388 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1452/2897] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:24:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:25:00.026888 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=47 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:25:04.592617 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=9 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:25:04.729529 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c095408-0aa3-402b-93cf-e78fbed7c5de service cinder] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:25:04.749095 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c095408-0aa3-402b-93cf-e78fbed7c5de service cinder] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:25:04.752697 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4f328-78a5-41e8-b2aa-97009e5a45ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:25:04.753370 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f328-78a5-41e8-b2aa-97009e5a45ac service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:25:04.753656 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f328-78a5-41e8-b2aa-97009e5a45ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:25:04.753934 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f328-78a5-41e8-b2aa-97009e5a45ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:25:04.755707 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-90b4f328-78a5-41e8-b2aa-97009e5a45ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:25:04.778617 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4f328-78a5-41e8-b2aa-97009e5a45ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:25:04.780043 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4f328-78a5-41e8-b2aa-97009e5a45ac service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:25:04.788206 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1446/2898] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:25:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:25:14.784404 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=15 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:26:36.831477 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce002e-9bf1-47ff-b9fd-1bb6ef83a87b service nova] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:26:36.840832 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce002e-9bf1-47ff-b9fd-1bb6ef83a87b service nova] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:26:36.844313 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4f30c-236d-4992-b2b3-05e11b07e1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:26:36.844973 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4f30c-236d-4992-b2b3-05e11b07e1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:26:36.845258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4f30c-236d-4992-b2b3-05e11b07e1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:26:36.845535 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4f30c-236d-4992-b2b3-05e11b07e1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:26:36.847343 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-a5c4f30c-236d-4992-b2b3-05e11b07e1a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:26:36.857744 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4f30c-236d-4992-b2b3-05e11b07e1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:26:36.859153 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4f30c-236d-4992-b2b3-05e11b07e1a0 service neutron] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:26:36.868042 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1453/2899] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:26:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:26:46.864088 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=6 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:26:51.234533 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4f328-78a5-41e8-b2aa-97009e5a45ac service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:26:51.235966 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-90b4f328-78a5-41e8-b2aa-97009e5a45ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:26:51.246507 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4f328-78a5-41e8-b2aa-97009e5a45ac service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:26:51.250100 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe10ad-d4d3-45c9-a614-4b4907447c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '887d2ad3cbf7459a8a59e7442ba9744e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:26:51.250731 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe10ad-d4d3-45c9-a614-4b4907447c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:26:51.251010 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe10ad-d4d3-45c9-a614-4b4907447c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:26:51.251281 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe10ad-d4d3-45c9-a614-4b4907447c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:26:51.253109 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-30fe10ad-d4d3-45c9-a614-4b4907447c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:26:51.336096 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe10ad-d4d3-45c9-a614-4b4907447c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:26:51.337545 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe10ad-d4d3-45c9-a614-4b4907447c5f service neutron] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:26:51.356556 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1447/2900] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:26:54.818267 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4f30c-236d-4992-b2b3-05e11b07e1a0 service neutron] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:26:54.830870 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4f30c-236d-4992-b2b3-05e11b07e1a0 service neutron] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:26:54.835865 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f4b37-f0e6-4327-99be-0079ab41467c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:26:54.836606 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4b37-f0e6-4327-99be-0079ab41467c service placement] REQUEST_METHOD: `GET` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:26:54.836978 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4b37-f0e6-4327-99be-0079ab41467c service placement] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:26:54.837318 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4b37-f0e6-4327-99be-0079ab41467c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:26:54.839189 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-fa5f4b37-f0e6-4327-99be-0079ab41467c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:26:54.851662 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f4b37-f0e6-4327-99be-0079ab41467c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:26:54.853349 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f4b37-f0e6-4327-99be-0079ab41467c service placement] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:26:54.864158 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1454/2901] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:26:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:26:58.383967 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe10ad-d4d3-45c9-a614-4b4907447c5f service neutron] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:26:58.385463 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-30fe10ad-d4d3-45c9-a614-4b4907447c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:26:58.399664 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe10ad-d4d3-45c9-a614-4b4907447c5f service neutron] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:26:58.404882 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:26:58.405745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:26:58.406026 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:26:58.406296 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] PATH_INFO: `/v3/users/b1cf6897a2a74b7c84b26bae868845f4` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:26:58.407516 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1cf6897a2a74b7c84b26bae868845f4)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:26:58.409380 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:26:58.546430 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1cf6897a2a74b7c84b26bae868845f4'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:58.553220 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1cf6897a2a74b7c84b26bae868845f4'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:58.553874 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1cf6897a2a74b7c84b26bae868845f4'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:58.566153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1cf6897a2a74b7c84b26bae868845f4'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:58.566810 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1cf6897a2a74b7c84b26bae868845f4'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:58.639166 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1448/2902] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:26:58 2022] DELETE /identity/v3/users/b1cf6897a2a74b7c84b26bae868845f4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:26:58.864347 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f4b37-f0e6-4327-99be-0079ab41467c service placement] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:26:58.876955 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f4b37-f0e6-4327-99be-0079ab41467c service placement] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:26:58.934503 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:26:58.935354 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:26:58.935630 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:26:58.935898 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] PATH_INFO: `/v3/projects/85bf8c1d40a04aaf9e46cd7f1e58048a` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:26:58.937079 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85bf8c1d40a04aaf9e46cd7f1e58048a)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:26:58.938961 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:26:59.044506 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85bf8c1d40a04aaf9e46cd7f1e58048a', 'request_id': 'req-083a5239-82bb-41ff-87e6-593d06902e46'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.049862 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85bf8c1d40a04aaf9e46cd7f1e58048a', 'request_id': 'req-083a5239-82bb-41ff-87e6-593d06902e46'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.055849 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85bf8c1d40a04aaf9e46cd7f1e58048a', 'request_id': 'req-083a5239-82bb-41ff-87e6-593d06902e46'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.129667 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] The token cache is being invalidate because project 85bf8c1d40a04aaf9e46cd7f1e58048a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:26:59.129950 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.130551 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.132075 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1455/2903] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:26:58 2022] DELETE /identity/v3/projects/85bf8c1d40a04aaf9e46cd7f1e58048a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:26:59.135972 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:26:59.137399 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:26:59.150835 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270638a9-158d-4eb5-b668-f4e17e971d93 admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:26:59.183628 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:26:59.207681 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:26:59.208552 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:26:59.208867 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:26:59.209141 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] PATH_INFO: `/v3/users/24f0920000684a17992bc96f90278317` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:26:59.210327 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f0920000684a17992bc96f90278317)` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:26:59.212235 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:26:59.411641 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f0920000684a17992bc96f90278317'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.418933 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f0920000684a17992bc96f90278317'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.419153 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f0920000684a17992bc96f90278317'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.430965 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f0920000684a17992bc96f90278317'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.431594 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.notifications [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f0920000684a17992bc96f90278317'} {{(pid=80397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.495532 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1449/2904] 10.222.0.45 () {58 vars in 1223 bytes} [Tue Jul 5 16:26:59 2022] DELETE /identity/v3/users/24f0920000684a17992bc96f90278317 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:26:59.754449 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] Authenticating user token {{(pid=80398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:26:59.756003 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.fernet_utils [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:26:59.770408 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a5239-82bb-41ff-87e6-593d06902e46 admin admin] Validating token access rules against request {{(pid=80398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:26:59.828437 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ceaef67f814339b45f2d297054bedf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35018c3fbb4f440c9fd930631a71e6c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:26:59.829333 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:26:59.829614 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:26:59.829886 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] PATH_INFO: `/v3/projects/62a2450c58494ce0a452354e69162dd2` {{(pid=80398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:26:59.831062 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a2450c58494ce0a452354e69162dd2)` {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:26:59.832991 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] RBAC: Authorization granted {{(pid=80398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:26:59.854556 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:26:59.946164 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a2450c58494ce0a452354e69162dd2', 'request_id': 'req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.951581 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a2450c58494ce0a452354e69162dd2', 'request_id': 'req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:26:59.957930 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a2450c58494ce0a452354e69162dd2', 'request_id': 'req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a'} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:27:00.038445 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] The token cache is being invalidate because project 62a2450c58494ce0a452354e69162dd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 16:27:00.038659 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:27:00.039258 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG keystone.notifications [None req-959f26d6-fbe1-45cf-85f3-8e1ce1faf02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 16:27:00.040823 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: [pid: 80398|app: 0|req: 1456/2905] 10.222.0.45 () {58 vars in 1232 bytes} [Tue Jul 5 16:26:59 2022] DELETE /identity/v3/projects/62a2450c58494ce0a452354e69162dd2 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 16:27:09.491749 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:27:10.142563 dv-cinder-848759-1-iscsi devstack@keystone.service[80398]: DEBUG dbcounter [-] [80398] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 16:27:11.936527 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] Authenticating user token {{(pid=80397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 16:27:11.937974 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:27:12.006543 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3620855-0b6e-4ae4-a049-c7a0ae0ca60a admin admin] Validating token access rules against request {{(pid=80397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 16:27:12.064705 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17199bed-100b-4701-92b7-570f1f7fad64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75b7f541993b4a7c915c9e6d440e2674', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc8da89a42c743bf8f71b3d7cf6ab6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 16:27:12.065383 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17199bed-100b-4701-92b7-570f1f7fad64 service placement] REQUEST_METHOD: `GET` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 16:27:12.065667 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17199bed-100b-4701-92b7-570f1f7fad64 service placement] SCRIPT_NAME: `/identity` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 16:27:12.065942 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17199bed-100b-4701-92b7-570f1f7fad64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 16:27:12.067745 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.fernet_utils [None req-17199bed-100b-4701-92b7-570f1f7fad64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 16:27:12.078825 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17199bed-100b-4701-92b7-570f1f7fad64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 16:27:12.080386 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17199bed-100b-4701-92b7-570f1f7fad64 service placement] RBAC: Authorization granted {{(pid=80397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 16:27:12.090690 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: [pid: 80397|app: 0|req: 1450/2906] 10.222.0.45 () {60 vars in 1365 bytes} [Tue Jul 5 16:27:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 16:27:22.085940 dv-cinder-848759-1-iscsi devstack@keystone.service[80397]: DEBUG dbcounter [-] [80397] Writing DB stats keystone:SELECT=58 {{(pid=80397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}